[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:55:19 dist-test-slave-zkpd krb5kdc[6725](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:55:19 dist-test-slave-zkpd krb5kdc[6725](info): set up 2 sockets
Aug 11 11:55:19 dist-test-slave-zkpd krb5kdc[6725](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 11:55:20.488869 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:55:20 dist-test-slave-zkpd krb5kdc[6725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:55:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:55:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
I20260811 11:55:20.613648 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:34263
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:43439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:34263
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:20.724799 6740 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:20.725109 6740 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:20.725193 6740 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:20.729384 6740 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:55:20.729504 6740 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:20.729568 6740 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:20.729604 6740 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:55:20.729625 6740 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:55:20.735682 6740 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:43439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:34263
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:34263
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6740
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:20.736737 6740 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:20.737802 6740 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:20.746376 6745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:20.746521 6740 server_base.cc:1034] running on GCE node
W20260811 11:55:20.746389 6746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:20.746392 6748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:20.747215 6740 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:20.748345 6740 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:20.749544 6740 hybrid_clock.cc:648] HybridClock initialized: now 1786449320749516 us; error 38 us; skew 500 ppm
Aug 11 11:55:20 dist-test-slave-zkpd krb5kdc[6725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449320, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:20.752318 6740 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:55:20.753154 6740 webserver.cc:460] Webserver started at http://127.6.143.190:43703/ using document root <none> and password file <none>
I20260811 11:55:20.753733 6740 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:20.753825 6740 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:20.754057 6740 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:20.756125 6740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "d7e999c034c5416dbdf883a07df4895b"
format_stamp: "Formatted at 2026-08-11 11:55:20 on dist-test-slave-zkpd"
server_key: "062ffbf8ef57be8720b0b8acd55dad80"
server_key_iv: "5e1e2803e449eee9cbb3527e90545aa4"
server_key_version: "encryptionkey@0"
I20260811 11:55:20.756698 6740 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "d7e999c034c5416dbdf883a07df4895b"
format_stamp: "Formatted at 2026-08-11 11:55:20 on dist-test-slave-zkpd"
server_key: "062ffbf8ef57be8720b0b8acd55dad80"
server_key_iv: "5e1e2803e449eee9cbb3527e90545aa4"
server_key_version: "encryptionkey@0"
I20260811 11:55:20.760426 6740 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 11:55:20.762734 6756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:20.763931 6740 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:55:20.764072 6740 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "d7e999c034c5416dbdf883a07df4895b"
format_stamp: "Formatted at 2026-08-11 11:55:20 on dist-test-slave-zkpd"
server_key: "062ffbf8ef57be8720b0b8acd55dad80"
server_key_iv: "5e1e2803e449eee9cbb3527e90545aa4"
server_key_version: "encryptionkey@0"
I20260811 11:55:20.764178 6740 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:20.776989 6740 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:20.780328 6740 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:20.780546 6740 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:20.787537 6740 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:34263
I20260811 11:55:20.787606 6810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:34263 every 8 connection(s)
I20260811 11:55:20.788527 6740 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:55:20.789988 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 6740
I20260811 11:55:20.790164 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:55:20.791481 6718 external_mini_cluster.cc:1428] Setting key 2c05d1d2c57d94ad0a9a9286ff7787aa
I20260811 11:55:20.791545 6811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:20.797467 6811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7e999c034c5416dbdf883a07df4895b: Bootstrap starting.
I20260811 11:55:20.799912 6811 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7e999c034c5416dbdf883a07df4895b: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:20.800814 6811 log.cc:824] T 00000000000000000000000000000000 P d7e999c034c5416dbdf883a07df4895b: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:20.802785 6811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7e999c034c5416dbdf883a07df4895b: No bootstrap required, opened a new log
I20260811 11:55:20.806809 6811 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7e999c034c5416dbdf883a07df4895b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e999c034c5416dbdf883a07df4895b" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 34263 } }
I20260811 11:55:20.807045 6811 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7e999c034c5416dbdf883a07df4895b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:20.807111 6811 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7e999c034c5416dbdf883a07df4895b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e999c034c5416dbdf883a07df4895b, State: Initialized, Role: FOLLOWER
I20260811 11:55:20.807550 6811 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7e999c034c5416dbdf883a07df4895b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e999c034c5416dbdf883a07df4895b" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 34263 } }
I20260811 11:55:20.807704 6811 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7e999c034c5416dbdf883a07df4895b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:55:20.807796 6811 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7e999c034c5416dbdf883a07df4895b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:55:20.807935 6811 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7e999c034c5416dbdf883a07df4895b [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:20.808837 6811 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7e999c034c5416dbdf883a07df4895b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e999c034c5416dbdf883a07df4895b" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 34263 } }
I20260811 11:55:20.809218 6811 leader_election.cc:302] T 00000000000000000000000000000000 P d7e999c034c5416dbdf883a07df4895b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7e999c034c5416dbdf883a07df4895b; no voters:
Aug 11 11:55:20 dist-test-slave-zkpd krb5kdc[6725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449320, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:20.809543 6811 leader_election.cc:288] T 00000000000000000000000000000000 P d7e999c034c5416dbdf883a07df4895b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:55:20.809655 6816 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7e999c034c5416dbdf883a07df4895b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:20.809887 6816 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7e999c034c5416dbdf883a07df4895b [term 1 LEADER]: Becoming Leader. State: Replica: d7e999c034c5416dbdf883a07df4895b, State: Running, Role: LEADER
I20260811 11:55:20.810277 6816 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7e999c034c5416dbdf883a07df4895b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e999c034c5416dbdf883a07df4895b" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 34263 } }
I20260811 11:55:20.810884 6811 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7e999c034c5416dbdf883a07df4895b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:55:20.812168 6817 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7e999c034c5416dbdf883a07df4895b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7e999c034c5416dbdf883a07df4895b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e999c034c5416dbdf883a07df4895b" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 34263 } } }
I20260811 11:55:20.812296 6817 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7e999c034c5416dbdf883a07df4895b [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:20.812674 6818 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7e999c034c5416dbdf883a07df4895b [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7e999c034c5416dbdf883a07df4895b. Latest consensus state: current_term: 1 leader_uuid: "d7e999c034c5416dbdf883a07df4895b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e999c034c5416dbdf883a07df4895b" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 34263 } } }
I20260811 11:55:20.812795 6818 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7e999c034c5416dbdf883a07df4895b [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:20.813534 6825 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:55:20.816817 6825 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:55:20.816977 6813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:20.794620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52470 (local address 127.6.143.190:34263)
0811 11:55:20.795130 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:20.795151 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:20.796333 (+ 1182us) server_negotiation.cc:407] Connection header received
0811 11:55:20.799488 (+ 3155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:20.799510 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:20.799931 (+ 421us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:20.800299 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:20.802363 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:20.803830 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:20.805041 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:20.805843 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:20.806014 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:20.810315 (+ 4301us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:20.810322 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:20.810337 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:20.810367 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:20.812600 (+ 2233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:20.813325 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:20.813331 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:20.813336 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:20.813471 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:20.813911 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:20.813915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:20.813916 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:20.814441 (+ 525us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:20.814650 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:20.815748 (+ 1098us) server_negotiation.cc:299] Negotiation successful
0811 11:55:20.816043 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":141,"threads_started":1}
I20260811 11:55:20.822388 6825 catalog_manager.cc:1347] Generated new cluster ID: 2d2db573fa05474cb3004d68b8a945cb
I20260811 11:55:20.822453 6825 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:55:20.836630 6825 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:55:20.837805 6825 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:55:20.851048 6825 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7e999c034c5416dbdf883a07df4895b: Generated new TSK 0
I20260811 11:55:20.851929 6825 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
I20260811 11:55:20.884119 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:0
--local_ip_for_outbound_sockets=127.6.143.129
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:34263
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:43439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:20.997740 6839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:20.998016 6839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:20.998101 6839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:21.002159 6839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:21.002238 6839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:21.002322 6839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:55:21.007483 6839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:43439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:34263
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:21.008546 6839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:21.009536 6839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:21.019343 6849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:21.019419 6839 server_base.cc:1034] running on GCE node
W20260811 11:55:21.019366 6847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:21.019369 6846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:21.020138 6839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:21.020749 6839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:21.021939 6839 hybrid_clock.cc:648] HybridClock initialized: now 1786449321021925 us; error 26 us; skew 500 ppm
Aug 11 11:55:21 dist-test-slave-zkpd krb5kdc[6725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449321, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:21.024614 6839 init.cc:377] Logged in from keytab as kudu/127.6.143.129@KRBTEST.COM (short username kudu)
I20260811 11:55:21.025358 6839 webserver.cc:460] Webserver started at http://127.6.143.129:34207/ using document root <none> and password file <none>
I20260811 11:55:21.025862 6839 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:21.025918 6839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:21.026072 6839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:21.028033 6839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-0/data/instance:
uuid: "ab11e07377164e648a49e87165000652"
format_stamp: "Formatted at 2026-08-11 11:55:21 on dist-test-slave-zkpd"
server_key: "37650dd60f8b5474d7d6b2c1e95260dd"
server_key_iv: "4f1f5f3831a8b8b99d33463385eb017e"
server_key_version: "encryptionkey@0"
I20260811 11:55:21.028550 6839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance:
uuid: "ab11e07377164e648a49e87165000652"
format_stamp: "Formatted at 2026-08-11 11:55:21 on dist-test-slave-zkpd"
server_key: "37650dd60f8b5474d7d6b2c1e95260dd"
server_key_iv: "4f1f5f3831a8b8b99d33463385eb017e"
server_key_version: "encryptionkey@0"
I20260811 11:55:21.032145 6839 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 11:55:21.034533 6856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:21.035820 6839 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:21.035996 6839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "ab11e07377164e648a49e87165000652"
format_stamp: "Formatted at 2026-08-11 11:55:21 on dist-test-slave-zkpd"
server_key: "37650dd60f8b5474d7d6b2c1e95260dd"
server_key_iv: "4f1f5f3831a8b8b99d33463385eb017e"
server_key_version: "encryptionkey@0"
I20260811 11:55:21.036113 6839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:21.059765 6839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:21.062633 6839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:21.062829 6839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:21.063419 6839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:55:21.064313 6839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:21.064363 6839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:21.064430 6839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:21.064462 6839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:21.073714 6839 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:42243
I20260811 11:55:21.073750 6978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:42243 every 8 connection(s)
I20260811 11:55:21.074577 6839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:55:21.080681 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 6839
I20260811 11:55:21.080818 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance
I20260811 11:55:21.081081 6718 external_mini_cluster.cc:1428] Setting key 1d4f27fc25a17e5efdfc98ebc3784af7
Aug 11 11:55:21 dist-test-slave-zkpd krb5kdc[6725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449321, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:21.088791 6813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:21.076370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:47711 (local address 127.6.143.190:34263)
0811 11:55:21.076511 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:21.076515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:21.077258 (+ 743us) server_negotiation.cc:407] Connection header received
0811 11:55:21.078674 (+ 1416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:21.078677 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:21.078735 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:21.078843 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:21.080045 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:21.080584 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:21.081949 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:21.082282 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:21.082399 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:21.085009 (+ 2610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:21.085012 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:21.085014 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:21.085037 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:21.086568 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:21.087098 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:21.087103 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:21.087105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:21.087189 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:21.087664 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:21.087666 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:21.087667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:21.087841 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:21.087980 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:21.088485 (+ 505us) server_negotiation.cc:299] Negotiation successful
0811 11:55:21.088626 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 11:55:21.089599 6981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:21.076669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:34263 (local address 127.6.143.129:47711)
0811 11:55:21.077115 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:21.077154 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:21.078461 (+ 1307us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:21.079023 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:21.079030 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:21.079357 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:21.079820 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:21.079846 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:21.080713 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:21.080717 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:21.081817 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:21.081829 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:21.082454 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:21.082459 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:21.082622 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:21.083188 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:21.083204 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:21.084840 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:21.086701 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:21.086707 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:21.086721 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:21.086987 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:21.087341 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:21.087349 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:21.087354 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:21.087557 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:21.087982 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:21.087987 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:21.088239 (+ 252us) client_negotiation.cc:769] Sending connection context
0811 11:55:21.088432 (+ 193us) client_negotiation.cc:240] Negotiation successful
0811 11:55:21.088748 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":117,"threads_started":1}
I20260811 11:55:21.090975 6979 heartbeater.cc:347] Connected to a master server at 127.6.143.190:34263
I20260811 11:55:21.091207 6979 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:21.091676 6979 heartbeater.cc:511] Master 127.6.143.190:34263 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.143.130@KRBTEST.COM; defaulting to no policy
I20260811 11:55:21.093327 6773 ts_manager.cc:194] Registered new tserver with Master: ab11e07377164e648a49e87165000652 (127.6.143.129:42243)
I20260811 11:55:21.094509 6773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.129@KRBTEST.COM'} at 127.6.143.129:47711
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
I20260811 11:55:21.111147 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:0
--local_ip_for_outbound_sockets=127.6.143.130
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:34263
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:43439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:21.223498 6984 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:21.223737 6984 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:21.223780 6984 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:21.227770 6984 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:21.227837 6984 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:21.227910 6984 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:55:21.233261 6984 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:43439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:34263
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6984
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:21.234246 6984 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:21.235189 6984 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 11:55:21.244971 6984 server_base.cc:1034] running on GCE node
W20260811 11:55:21.244954 6989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:21.244976 6992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:21.244935 6990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:21.245623 6984 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:21.246177 6984 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:21.247357 6984 hybrid_clock.cc:648] HybridClock initialized: now 1786449321247331 us; error 50 us; skew 500 ppm
Aug 11 11:55:21 dist-test-slave-zkpd krb5kdc[6725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449321, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:21.250074 6984 init.cc:377] Logged in from keytab as kudu/127.6.143.130@KRBTEST.COM (short username kudu)
I20260811 11:55:21.250926 6984 webserver.cc:460] Webserver started at http://127.6.143.130:42335/ using document root <none> and password file <none>
I20260811 11:55:21.251475 6984 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:21.251566 6984 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:21.251793 6984 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:21.253801 6984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-1/data/instance:
uuid: "5377a738b3fb4180beb19d77210e1bb9"
format_stamp: "Formatted at 2026-08-11 11:55:21 on dist-test-slave-zkpd"
server_key: "a04a8c0af0ffc5b47ff7dcf9b30f9170"
server_key_iv: "2522568c6d1e11cad6e2f3622b018190"
server_key_version: "encryptionkey@0"
I20260811 11:55:21.254376 6984 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance:
uuid: "5377a738b3fb4180beb19d77210e1bb9"
format_stamp: "Formatted at 2026-08-11 11:55:21 on dist-test-slave-zkpd"
server_key: "a04a8c0af0ffc5b47ff7dcf9b30f9170"
server_key_iv: "2522568c6d1e11cad6e2f3622b018190"
server_key_version: "encryptionkey@0"
I20260811 11:55:21.257972 6984 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 11:55:21.260251 7000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:21.261418 6984 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:55:21.261556 6984 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "5377a738b3fb4180beb19d77210e1bb9"
format_stamp: "Formatted at 2026-08-11 11:55:21 on dist-test-slave-zkpd"
server_key: "a04a8c0af0ffc5b47ff7dcf9b30f9170"
server_key_iv: "2522568c6d1e11cad6e2f3622b018190"
server_key_version: "encryptionkey@0"
I20260811 11:55:21.261662 6984 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:21.278473 6984 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:21.281667 6984 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:21.281867 6984 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:21.282461 6984 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:55:21.283599 6984 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:21.283671 6984 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:21.283739 6984 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:21.283788 6984 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:21.292712 6984 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:42019
I20260811 11:55:21.292735 7120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:42019 every 8 connection(s)
I20260811 11:55:21.293561 6984 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
I20260811 11:55:21.297868 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 6984
I20260811 11:55:21.298002 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance
I20260811 11:55:21.298275 6718 external_mini_cluster.cc:1428] Setting key 8a60a620dad5ef9e55ddf6d39925bb5a
Aug 11 11:55:21 dist-test-slave-zkpd krb5kdc[6725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449321, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:21.308946 6813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:21.295384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:51499 (local address 127.6.143.190:34263)
0811 11:55:21.295534 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:21.295537 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:21.296438 (+ 901us) server_negotiation.cc:407] Connection header received
0811 11:55:21.297804 (+ 1366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:21.297807 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:21.297866 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:21.297979 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:21.299429 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:21.300086 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:21.301813 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:21.302225 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:21.302346 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:21.305043 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:21.305047 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:21.305049 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:21.305081 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:21.306785 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:21.307271 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:21.307275 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:21.307277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:21.307373 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:21.307715 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:21.307719 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:21.307721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:21.307997 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:21.308112 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:21.308576 (+ 464us) server_negotiation.cc:299] Negotiation successful
0811 11:55:21.308710 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
WARNING: no policy specified for kudu/127.6.143.131@KRBTEST.COM; defaulting to no policy
I20260811 11:55:21.309626 7123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:21.295772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:34263 (local address 127.6.143.130:51499)
0811 11:55:21.296284 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:21.296326 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:21.297587 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:21.298205 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:21.298215 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:21.298689 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:21.299276 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:21.299296 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:21.300216 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:21.300220 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:21.301680 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:21.301696 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:21.302380 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:21.302383 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:21.302528 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:21.303106 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:21.303122 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:21.304867 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:21.306902 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:21.306907 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:21.306919 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:21.307165 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:21.307483 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:21.307485 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:21.307487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:21.307615 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:21.308101 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:21.308106 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:21.308320 (+ 214us) client_negotiation.cc:769] Sending connection context
0811 11:55:21.308483 (+ 163us) client_negotiation.cc:240] Negotiation successful
0811 11:55:21.308723 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":182,"threads_started":1}
I20260811 11:55:21.310811 7121 heartbeater.cc:347] Connected to a master server at 127.6.143.190:34263
I20260811 11:55:21.311060 7121 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:21.311667 7121 heartbeater.cc:511] Master 127.6.143.190:34263 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.131@KRBTEST.COM" created.
I20260811 11:55:21.312942 6773 ts_manager.cc:194] Registered new tserver with Master: 5377a738b3fb4180beb19d77210e1bb9 (127.6.143.130:42019)
I20260811 11:55:21.313581 6773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.130@KRBTEST.COM'} at 127.6.143.130:51499
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
I20260811 11:55:21.326735 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:0
--local_ip_for_outbound_sockets=127.6.143.131
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:34263
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:43439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:21.440032 7126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:21.440260 7126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:21.440300 7126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:21.444260 7126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:21.444308 7126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:21.444371 7126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:55:21.449283 7126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:43439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:34263
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:21.450170 7126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:21.451076 7126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:21.461375 7131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:21.461741 7126 server_base.cc:1034] running on GCE node
W20260811 11:55:21.462133 7132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:21.462180 7134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:21.462617 7126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:21.463316 7126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:21.464498 7126 hybrid_clock.cc:648] HybridClock initialized: now 1786449321464467 us; error 44 us; skew 500 ppm
Aug 11 11:55:21 dist-test-slave-zkpd krb5kdc[6725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449321, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:21.468017 7126 init.cc:377] Logged in from keytab as kudu/127.6.143.131@KRBTEST.COM (short username kudu)
I20260811 11:55:21.468927 7126 webserver.cc:460] Webserver started at http://127.6.143.131:36105/ using document root <none> and password file <none>
I20260811 11:55:21.469460 7126 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:21.469520 7126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:21.469681 7126 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:21.471668 7126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-2/data/instance:
uuid: "f25b378edcc3435d827c065a555b6167"
format_stamp: "Formatted at 2026-08-11 11:55:21 on dist-test-slave-zkpd"
server_key: "dbb1f8c263a3c080d83981dec4ed2e4f"
server_key_iv: "24d4635481ee3dd52c19ddf6fec882ff"
server_key_version: "encryptionkey@0"
I20260811 11:55:21.472250 7126 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance:
uuid: "f25b378edcc3435d827c065a555b6167"
format_stamp: "Formatted at 2026-08-11 11:55:21 on dist-test-slave-zkpd"
server_key: "dbb1f8c263a3c080d83981dec4ed2e4f"
server_key_iv: "24d4635481ee3dd52c19ddf6fec882ff"
server_key_version: "encryptionkey@0"
I20260811 11:55:21.475865 7126 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 11:55:21.478268 7141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:21.479504 7126 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:21.479642 7126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "f25b378edcc3435d827c065a555b6167"
format_stamp: "Formatted at 2026-08-11 11:55:21 on dist-test-slave-zkpd"
server_key: "dbb1f8c263a3c080d83981dec4ed2e4f"
server_key_iv: "24d4635481ee3dd52c19ddf6fec882ff"
server_key_version: "encryptionkey@0"
I20260811 11:55:21.479753 7126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:21.496045 7126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:21.498792 7126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:21.499003 7126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:21.499580 7126 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:55:21.500500 7126 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:21.500573 7126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:21.500650 7126 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:21.500694 7126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:21.509943 7126 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:33183
I20260811 11:55:21.510026 7265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:33183 every 8 connection(s)
I20260811 11:55:21.510916 7126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:55:21.512889 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 7126
I20260811 11:55:21.513018 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance
I20260811 11:55:21.513293 6718 external_mini_cluster.cc:1428] Setting key f19bd2e84989eaaaf213abf4eec70465
Aug 11 11:55:21 dist-test-slave-zkpd krb5kdc[6725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449321, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:21.526850 6813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:21.512941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:39155 (local address 127.6.143.190:34263)
0811 11:55:21.513097 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:21.513103 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:21.514121 (+ 1018us) server_negotiation.cc:407] Connection header received
0811 11:55:21.515505 (+ 1384us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:21.515511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:21.515579 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:21.515670 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:21.517172 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:21.517822 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:21.519654 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:21.519870 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:21.519998 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:21.522710 (+ 2712us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:21.522717 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:21.522723 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:21.522761 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:21.524814 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:21.525334 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:21.525338 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:21.525340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:21.525430 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:21.525776 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:21.525781 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:21.525783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:21.525976 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:21.526084 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:21.526566 (+ 482us) server_negotiation.cc:299] Negotiation successful
0811 11:55:21.526682 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 11:55:21.527597 7268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:21.513326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:34263 (local address 127.6.143.131:39155)
0811 11:55:21.513950 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:21.514007 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:21.515291 (+ 1284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:21.515826 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:21.515833 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:21.516378 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:21.517027 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:21.517045 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:21.517962 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:21.517969 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:21.519515 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:21.519531 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:21.520045 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:21.520052 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:21.520292 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:21.520897 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:21.520933 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:21.522455 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:21.524937 (+ 2482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:21.524946 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:21.524960 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:21.525219 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:21.525521 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:21.525525 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:21.525527 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:21.525670 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:21.526084 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:21.526091 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:21.526330 (+ 239us) client_negotiation.cc:769] Sending connection context
0811 11:55:21.526522 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 11:55:21.526828 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":141,"threads_started":1}
I20260811 11:55:21.528868 7266 heartbeater.cc:347] Connected to a master server at 127.6.143.190:34263
I20260811 11:55:21.529116 7266 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:21.529707 7266 heartbeater.cc:511] Master 127.6.143.190:34263 requested a full tablet report, sending...
I20260811 11:55:21.530722 6773 ts_manager.cc:194] Registered new tserver with Master: f25b378edcc3435d827c065a555b6167 (127.6.143.131:33183)
I20260811 11:55:21.531283 6773 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.131@KRBTEST.COM'} at 127.6.143.131:39155
I20260811 11:55:21.541419 6718 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 11:55:21 dist-test-slave-zkpd krb5kdc[6725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449321, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 11:55:21 dist-test-slave-zkpd krb5kdc[6725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449321, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.143.129@KRBTEST.COM
I20260811 11:55:21.568460 7273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:21.558865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32862 (local address 127.6.143.129:42243)
0811 11:55:21.559175 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:21.559192 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:21.559219 (+ 27us) server_negotiation.cc:407] Connection header received
0811 11:55:21.559355 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:21.559367 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:21.559577 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:21.559729 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:21.560467 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:21.561243 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:21.562082 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:21.562464 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:21.562582 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:21.565027 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:21.565036 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:21.565049 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:21.565073 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:21.566793 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:21.567179 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:21.567184 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:21.567185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:21.567263 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:21.567514 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:21.567517 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:21.567521 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:21.567779 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:21.568010 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:21.568173 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 11:55:21.568299 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":86,"threads_started":1}
W20260811 11:55:21.569439 6888 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32862
Aug 11 11:55:21 dist-test-slave-zkpd krb5kdc[6725](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 11:55:21 dist-test-slave-zkpd krb5kdc[6725](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.129@KRBTEST.COM
I20260811 11:55:21.595726 7273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:21.587559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32870 (local address 127.6.143.129:42243)
0811 11:55:21.587729 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:21.587734 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:21.587875 (+ 141us) server_negotiation.cc:407] Connection header received
0811 11:55:21.588084 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:21.588089 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:21.588140 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:21.588226 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:21.588940 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:21.589463 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:21.590285 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:21.590525 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:21.590638 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:21.592965 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:21.592968 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:21.592970 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:21.592991 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:21.594414 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:21.594766 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:21.594769 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:21.594770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:21.594839 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:21.595077 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:21.595079 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:21.595080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:21.595226 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:21.595319 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:21.595513 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 11:55:21.595607 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 11:55:21.596774 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 6839
I20260811 11:55:21.604939 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 6984
I20260811 11:55:21.613327 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 7126
I20260811 11:55:21.620355 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 6740
2026-08-11T11:55:21Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2236 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:55:21 dist-test-slave-zkpd krb5kdc[7280](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:55:21 dist-test-slave-zkpd krb5kdc[7280](info): set up 2 sockets
Aug 11 11:55:21 dist-test-slave-zkpd krb5kdc[7280](info): commencing operation
krb5kdc: starting...
W20260811 11:55:22.718061 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:55:22 dist-test-slave-zkpd krb5kdc[7280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:55:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:55:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
I20260811 11:55:22.842167 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:35819
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:45491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:35819
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:22.953859 7293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:22.954123 7293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:22.954162 7293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:22.958021 7293 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:55:22.958072 7293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:22.958091 7293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:22.958108 7293 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:55:22.958123 7293 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:55:22.963248 7293 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:45491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:35819
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:35819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:22.964254 7293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:22.965237 7293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:22.973727 7299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:22.973703 7298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:22.973707 7301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:22.973902 7293 server_base.cc:1034] running on GCE node
I20260811 11:55:22.974618 7293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:22.975625 7293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:22.976801 7293 hybrid_clock.cc:648] HybridClock initialized: now 1786449322976781 us; error 30 us; skew 500 ppm
Aug 11 11:55:22 dist-test-slave-zkpd krb5kdc[7280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449322, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:22.979636 7293 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:55:22.980582 7293 webserver.cc:460] Webserver started at http://127.6.143.190:46733/ using document root <none> and password file <none>
I20260811 11:55:22.981168 7293 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:22.981235 7293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:22.981462 7293 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:22.983503 7293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "dad0653034a2403cb5db08ee473c2bcf"
format_stamp: "Formatted at 2026-08-11 11:55:22 on dist-test-slave-zkpd"
server_key: "603382b1270708e1975245afebd85260"
server_key_iv: "e56bddbb4758d3dc1af227f4ecfa905d"
server_key_version: "encryptionkey@0"
I20260811 11:55:22.984123 7293 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "dad0653034a2403cb5db08ee473c2bcf"
format_stamp: "Formatted at 2026-08-11 11:55:22 on dist-test-slave-zkpd"
server_key: "603382b1270708e1975245afebd85260"
server_key_iv: "e56bddbb4758d3dc1af227f4ecfa905d"
server_key_version: "encryptionkey@0"
I20260811 11:55:22.987700 7293 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 11:55:22.990144 7309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:22.991348 7293 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:22.991494 7293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "dad0653034a2403cb5db08ee473c2bcf"
format_stamp: "Formatted at 2026-08-11 11:55:22 on dist-test-slave-zkpd"
server_key: "603382b1270708e1975245afebd85260"
server_key_iv: "e56bddbb4758d3dc1af227f4ecfa905d"
server_key_version: "encryptionkey@0"
I20260811 11:55:22.991607 7293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:23.011834 7293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:23.014664 7293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:23.014895 7293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:23.022049 7293 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:35819
I20260811 11:55:23.022060 7364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:35819 every 8 connection(s)
I20260811 11:55:23.023002 7293 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:55:23.025876 7366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:23.028710 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 7293
I20260811 11:55:23.028843 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:55:23.029089 6718 external_mini_cluster.cc:1428] Setting key 4a19a89b0d2d22cbbd786f85c1f2784a
I20260811 11:55:23.031952 7366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dad0653034a2403cb5db08ee473c2bcf: Bootstrap starting.
I20260811 11:55:23.034149 7366 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dad0653034a2403cb5db08ee473c2bcf: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:23.034885 7366 log.cc:824] T 00000000000000000000000000000000 P dad0653034a2403cb5db08ee473c2bcf: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:23.036759 7366 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dad0653034a2403cb5db08ee473c2bcf: No bootstrap required, opened a new log
Aug 11 11:55:23 dist-test-slave-zkpd krb5kdc[7280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:23.040020 7366 raft_consensus.cc:348] T 00000000000000000000000000000000 P dad0653034a2403cb5db08ee473c2bcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dad0653034a2403cb5db08ee473c2bcf" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 35819 } }
I20260811 11:55:23.040256 7366 raft_consensus.cc:374] T 00000000000000000000000000000000 P dad0653034a2403cb5db08ee473c2bcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:23.040335 7366 raft_consensus.cc:729] T 00000000000000000000000000000000 P dad0653034a2403cb5db08ee473c2bcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dad0653034a2403cb5db08ee473c2bcf, State: Initialized, Role: FOLLOWER
I20260811 11:55:23.040766 7366 consensus_queue.cc:260] T 00000000000000000000000000000000 P dad0653034a2403cb5db08ee473c2bcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dad0653034a2403cb5db08ee473c2bcf" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 35819 } }
I20260811 11:55:23.040983 7366 raft_consensus.cc:388] T 00000000000000000000000000000000 P dad0653034a2403cb5db08ee473c2bcf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:55:23.041059 7366 raft_consensus.cc:482] T 00000000000000000000000000000000 P dad0653034a2403cb5db08ee473c2bcf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:55:23.041159 7366 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dad0653034a2403cb5db08ee473c2bcf [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:23.042176 7366 raft_consensus.cc:504] T 00000000000000000000000000000000 P dad0653034a2403cb5db08ee473c2bcf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dad0653034a2403cb5db08ee473c2bcf" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 35819 } }
I20260811 11:55:23.042570 7366 leader_election.cc:302] T 00000000000000000000000000000000 P dad0653034a2403cb5db08ee473c2bcf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dad0653034a2403cb5db08ee473c2bcf; no voters:
I20260811 11:55:23.042914 7366 leader_election.cc:288] T 00000000000000000000000000000000 P dad0653034a2403cb5db08ee473c2bcf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:55:23.043010 7371 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dad0653034a2403cb5db08ee473c2bcf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:23.043228 7371 raft_consensus.cc:686] T 00000000000000000000000000000000 P dad0653034a2403cb5db08ee473c2bcf [term 1 LEADER]: Becoming Leader. State: Replica: dad0653034a2403cb5db08ee473c2bcf, State: Running, Role: LEADER
I20260811 11:55:23.043604 7371 consensus_queue.cc:237] T 00000000000000000000000000000000 P dad0653034a2403cb5db08ee473c2bcf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dad0653034a2403cb5db08ee473c2bcf" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 35819 } }
I20260811 11:55:23.044176 7369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:23.030418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53528 (local address 127.6.143.190:35819)
0811 11:55:23.030845 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:23.030868 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:23.030902 (+ 34us) server_negotiation.cc:407] Connection header received
0811 11:55:23.031492 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:23.031509 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:23.031826 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:23.032223 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:23.033058 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.034056 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.034963 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.035468 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.035656 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:23.038179 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:23.038184 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:23.038196 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:23.038222 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:23.040730 (+ 2508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:23.041351 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:23.041357 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:23.041363 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:23.041475 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:23.041798 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:23.041801 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:23.041802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:23.042177 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:23.042375 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:23.042638 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 11:55:23.043181 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":118,"threads_started":1}
I20260811 11:55:23.044206 7366 sys_catalog.cc:564] T 00000000000000000000000000000000 P dad0653034a2403cb5db08ee473c2bcf [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:55:23.045259 7373 sys_catalog.cc:455] T 00000000000000000000000000000000 P dad0653034a2403cb5db08ee473c2bcf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dad0653034a2403cb5db08ee473c2bcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dad0653034a2403cb5db08ee473c2bcf" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 35819 } } }
I20260811 11:55:23.045396 7373 sys_catalog.cc:458] T 00000000000000000000000000000000 P dad0653034a2403cb5db08ee473c2bcf [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:23.045624 7373 sys_catalog.cc:455] T 00000000000000000000000000000000 P dad0653034a2403cb5db08ee473c2bcf [sys.catalog]: SysCatalogTable state changed. Reason: New leader dad0653034a2403cb5db08ee473c2bcf. Latest consensus state: current_term: 1 leader_uuid: "dad0653034a2403cb5db08ee473c2bcf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dad0653034a2403cb5db08ee473c2bcf" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 35819 } } }
I20260811 11:55:23.045708 7373 sys_catalog.cc:458] T 00000000000000000000000000000000 P dad0653034a2403cb5db08ee473c2bcf [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:23.046792 7381 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:55:23.049521 7381 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:55:23.055167 7381 catalog_manager.cc:1347] Generated new cluster ID: 715816417eaa4132bd60973b8330983a
I20260811 11:55:23.055228 7381 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:55:23.085417 7381 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:55:23.086417 7381 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:55:23.089838 7381 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dad0653034a2403cb5db08ee473c2bcf: Generated new TSK 0
I20260811 11:55:23.090520 7381 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
I20260811 11:55:23.121008 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:0
--local_ip_for_outbound_sockets=127.6.143.129
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:35819
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:45491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:23.233634 7394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:23.233930 7394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:23.233999 7394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:23.238035 7394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:23.238109 7394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:23.238194 7394 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:55:23.243511 7394 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:45491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:35819
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:23.244642 7394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:23.245713 7394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:23.255072 7401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:23.255072 7404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:23.255095 7402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:23.255250 7394 server_base.cc:1034] running on GCE node
I20260811 11:55:23.255659 7394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:23.256353 7394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:23.257541 7394 hybrid_clock.cc:648] HybridClock initialized: now 1786449323257514 us; error 46 us; skew 500 ppm
Aug 11 11:55:23 dist-test-slave-zkpd krb5kdc[7280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449323, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:23.260288 7394 init.cc:377] Logged in from keytab as kudu/127.6.143.129@KRBTEST.COM (short username kudu)
I20260811 11:55:23.261133 7394 webserver.cc:460] Webserver started at http://127.6.143.129:39195/ using document root <none> and password file <none>
I20260811 11:55:23.261649 7394 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:23.261708 7394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:23.261868 7394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:23.263768 7394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-0/data/instance:
uuid: "a4f782977957425f99d5cfb954de2ac6"
format_stamp: "Formatted at 2026-08-11 11:55:23 on dist-test-slave-zkpd"
server_key: "9688ab2df16cb5beddd16b7f4d262ef1"
server_key_iv: "f7c63cd1cba3fb9884fa72850e9e8405"
server_key_version: "encryptionkey@0"
I20260811 11:55:23.264353 7394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance:
uuid: "a4f782977957425f99d5cfb954de2ac6"
format_stamp: "Formatted at 2026-08-11 11:55:23 on dist-test-slave-zkpd"
server_key: "9688ab2df16cb5beddd16b7f4d262ef1"
server_key_iv: "f7c63cd1cba3fb9884fa72850e9e8405"
server_key_version: "encryptionkey@0"
I20260811 11:55:23.267812 7394 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 11:55:23.270170 7412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:23.271358 7394 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:23.271474 7394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "a4f782977957425f99d5cfb954de2ac6"
format_stamp: "Formatted at 2026-08-11 11:55:23 on dist-test-slave-zkpd"
server_key: "9688ab2df16cb5beddd16b7f4d262ef1"
server_key_iv: "f7c63cd1cba3fb9884fa72850e9e8405"
server_key_version: "encryptionkey@0"
I20260811 11:55:23.271569 7394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:23.288493 7394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:23.291358 7394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:23.291564 7394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:23.292196 7394 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:55:23.293109 7394 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:23.293175 7394 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:23.293241 7394 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:23.293283 7394 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:23.302510 7394 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:43541
I20260811 11:55:23.302531 7534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:43541 every 8 connection(s)
I20260811 11:55:23.303386 7394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:55:23.307504 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 7394
I20260811 11:55:23.307601 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance
I20260811 11:55:23.307861 6718 external_mini_cluster.cc:1428] Setting key bca28107db469f94f7fb4155670c04db
Aug 11 11:55:23 dist-test-slave-zkpd krb5kdc[7280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449323, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.143.130@KRBTEST.COM; defaulting to no policy
I20260811 11:55:23.319514 7369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:23.305262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:38851 (local address 127.6.143.190:35819)
0811 11:55:23.305423 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:23.305428 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:23.306478 (+ 1050us) server_negotiation.cc:407] Connection header received
0811 11:55:23.308509 (+ 2031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:23.308512 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:23.308568 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:23.308672 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:23.310112 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.310957 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.312729 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.312954 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.313033 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:23.315962 (+ 2929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:23.315966 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:23.315970 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:23.315998 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:23.317433 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:23.317948 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:23.317953 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:23.317955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:23.318038 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:23.318385 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:23.318390 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:23.318392 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:23.318617 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:23.318734 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:23.319219 (+ 485us) server_negotiation.cc:299] Negotiation successful
0811 11:55:23.319345 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 11:55:23.320369 7537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:23.305706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:35819 (local address 127.6.143.129:38851)
0811 11:55:23.306290 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:23.306357 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:23.308249 (+ 1892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:23.308883 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:23.308892 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:23.309427 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:23.309948 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:23.309963 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.311099 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.311102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:23.312565 (+ 1463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:23.312581 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.313125 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.313132 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:23.313366 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:23.314003 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:23.314030 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:23.315780 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:23.317564 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:23.317570 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:23.317585 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:23.317843 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:23.318132 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:23.318136 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:23.318138 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:23.318286 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:23.318726 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:23.318733 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:23.318982 (+ 249us) client_negotiation.cc:769] Sending connection context
0811 11:55:23.319175 (+ 193us) client_negotiation.cc:240] Negotiation successful
0811 11:55:23.319500 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":166,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.130@KRBTEST.COM" created.
I20260811 11:55:23.321882 7535 heartbeater.cc:347] Connected to a master server at 127.6.143.190:35819
I20260811 11:55:23.322140 7535 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:23.322641 7535 heartbeater.cc:511] Master 127.6.143.190:35819 requested a full tablet report, sending...
I20260811 11:55:23.324357 7326 ts_manager.cc:194] Registered new tserver with Master: a4f782977957425f99d5cfb954de2ac6 (127.6.143.129:43541)
I20260811 11:55:23.325613 7326 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.129@KRBTEST.COM'} at 127.6.143.129:38851
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
I20260811 11:55:23.337066 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:0
--local_ip_for_outbound_sockets=127.6.143.130
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:35819
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:45491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:23.451752 7540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:23.452004 7540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:23.452046 7540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:23.455839 7540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:23.455907 7540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:23.455976 7540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:55:23.463709 7540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:45491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:35819
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:23.465090 7540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:23.466279 7540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:23.476426 7551 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:23.476426 7548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:23.476435 7546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:23.476727 7540 server_base.cc:1034] running on GCE node
I20260811 11:55:23.477186 7540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:23.477824 7540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:23.479019 7540 hybrid_clock.cc:648] HybridClock initialized: now 1786449323478990 us; error 51 us; skew 500 ppm
Aug 11 11:55:23 dist-test-slave-zkpd krb5kdc[7280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449323, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:23.481771 7540 init.cc:377] Logged in from keytab as kudu/127.6.143.130@KRBTEST.COM (short username kudu)
I20260811 11:55:23.482668 7540 webserver.cc:460] Webserver started at http://127.6.143.130:40145/ using document root <none> and password file <none>
I20260811 11:55:23.483240 7540 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:23.483331 7540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:23.483564 7540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:23.485659 7540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-1/data/instance:
uuid: "3b97d16c560c408fb89383dfdd55987a"
format_stamp: "Formatted at 2026-08-11 11:55:23 on dist-test-slave-zkpd"
server_key: "bd5a2bc8c1ac1cd96673d22dc9492f5c"
server_key_iv: "00978b24292dcb17455067d099433532"
server_key_version: "encryptionkey@0"
I20260811 11:55:23.486274 7540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance:
uuid: "3b97d16c560c408fb89383dfdd55987a"
format_stamp: "Formatted at 2026-08-11 11:55:23 on dist-test-slave-zkpd"
server_key: "bd5a2bc8c1ac1cd96673d22dc9492f5c"
server_key_iv: "00978b24292dcb17455067d099433532"
server_key_version: "encryptionkey@0"
I20260811 11:55:23.489928 7540 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 11:55:23.492332 7558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:23.493528 7540 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:55:23.493685 7540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "3b97d16c560c408fb89383dfdd55987a"
format_stamp: "Formatted at 2026-08-11 11:55:23 on dist-test-slave-zkpd"
server_key: "bd5a2bc8c1ac1cd96673d22dc9492f5c"
server_key_iv: "00978b24292dcb17455067d099433532"
server_key_version: "encryptionkey@0"
I20260811 11:55:23.493795 7540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:23.539083 7540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:23.542351 7540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:23.542557 7540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:23.543196 7540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:55:23.544160 7540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:23.544243 7540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:23.544312 7540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:23.544358 7540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:23.553629 7540 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:43177
I20260811 11:55:23.553655 7676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:43177 every 8 connection(s)
I20260811 11:55:23.554638 7540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
Aug 11 11:55:23 dist-test-slave-zkpd krb5kdc[7280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449323, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:23.564710 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 7540
I20260811 11:55:23.564841 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance
I20260811 11:55:23.565210 6718 external_mini_cluster.cc:1428] Setting key 977001e2eb8636f34c59f807e3630576
I20260811 11:55:23.568485 7369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:23.556518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:54009 (local address 127.6.143.190:35819)
0811 11:55:23.556658 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:23.556662 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:23.557499 (+ 837us) server_negotiation.cc:407] Connection header received
0811 11:55:23.558866 (+ 1367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:23.558870 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:23.558930 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:23.559020 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:23.560321 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.560874 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.562030 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.562263 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.562385 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:23.564731 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:23.564734 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:23.564736 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:23.564762 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:23.566256 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:23.566874 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:23.566878 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:23.566880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:23.566966 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:23.567309 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:23.567314 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:23.567315 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:23.567515 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:23.567643 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:23.568202 (+ 559us) server_negotiation.cc:299] Negotiation successful
0811 11:55:23.568327 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 11:55:23.569382 7679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:23.556893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:35819 (local address 127.6.143.130:54009)
0811 11:55:23.557350 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:23.557396 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:23.558642 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:23.559206 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:23.559214 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:23.559577 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:23.560168 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:23.560185 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.561016 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.561020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:23.561894 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:23.561905 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.562433 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.562437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:23.562604 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:23.563135 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:23.563152 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:23.564555 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:23.566385 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:23.566391 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:23.566411 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:23.566761 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:23.567062 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:23.567066 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:23.567068 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:23.567211 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:23.567642 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:23.567650 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:23.567946 (+ 296us) client_negotiation.cc:769] Sending connection context
0811 11:55:23.568155 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 11:55:23.568484 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":110,"threads_started":1}
I20260811 11:55:23.570564 7677 heartbeater.cc:347] Connected to a master server at 127.6.143.190:35819
I20260811 11:55:23.570824 7677 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:23.571445 7677 heartbeater.cc:511] Master 127.6.143.190:35819 requested a full tablet report, sending...
I20260811 11:55:23.572677 7326 ts_manager.cc:194] Registered new tserver with Master: 3b97d16c560c408fb89383dfdd55987a (127.6.143.130:43177)
I20260811 11:55:23.573243 7326 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.130@KRBTEST.COM'} at 127.6.143.130:54009
WARNING: no policy specified for kudu/127.6.143.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
I20260811 11:55:23.592873 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:0
--local_ip_for_outbound_sockets=127.6.143.131
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:35819
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:45491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:23.706189 7682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:23.706452 7682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:23.706550 7682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:23.710578 7682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:23.710655 7682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:23.710765 7682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:55:23.715752 7682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:45491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:35819
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:23.716756 7682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:23.717746 7682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:23.726979 7687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:23.727067 7682 server_base.cc:1034] running on GCE node
W20260811 11:55:23.726989 7692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:23.727031 7688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:23.727655 7682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:23.728317 7682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:23.729485 7682 hybrid_clock.cc:648] HybridClock initialized: now 1786449323729466 us; error 30 us; skew 500 ppm
Aug 11 11:55:23 dist-test-slave-zkpd krb5kdc[7280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449323, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:23.732157 7682 init.cc:377] Logged in from keytab as kudu/127.6.143.131@KRBTEST.COM (short username kudu)
I20260811 11:55:23.732991 7682 webserver.cc:460] Webserver started at http://127.6.143.131:34517/ using document root <none> and password file <none>
I20260811 11:55:23.733559 7682 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:23.733652 7682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:23.733902 7682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:23.735965 7682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-2/data/instance:
uuid: "b02b4c7a95c54631a193dcf7551a9a98"
format_stamp: "Formatted at 2026-08-11 11:55:23 on dist-test-slave-zkpd"
server_key: "178929b86935dfcfd89a17d0ed94fdbd"
server_key_iv: "5ce04cf40f59dd7f91125cc04ffe7b55"
server_key_version: "encryptionkey@0"
I20260811 11:55:23.736552 7682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance:
uuid: "b02b4c7a95c54631a193dcf7551a9a98"
format_stamp: "Formatted at 2026-08-11 11:55:23 on dist-test-slave-zkpd"
server_key: "178929b86935dfcfd89a17d0ed94fdbd"
server_key_iv: "5ce04cf40f59dd7f91125cc04ffe7b55"
server_key_version: "encryptionkey@0"
I20260811 11:55:23.740209 7682 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 11:55:23.743431 7700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:23.744872 7682 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 11:55:23.745015 7682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "b02b4c7a95c54631a193dcf7551a9a98"
format_stamp: "Formatted at 2026-08-11 11:55:23 on dist-test-slave-zkpd"
server_key: "178929b86935dfcfd89a17d0ed94fdbd"
server_key_iv: "5ce04cf40f59dd7f91125cc04ffe7b55"
server_key_version: "encryptionkey@0"
I20260811 11:55:23.745127 7682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:23.771651 7682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:23.774617 7682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:23.774817 7682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:23.775480 7682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:55:23.776539 7682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:23.776609 7682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:23.776676 7682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:23.776726 7682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:23.785849 7682 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:34659
I20260811 11:55:23.785876 7817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:34659 every 8 connection(s)
I20260811 11:55:23.786732 7682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:55:23.789815 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 7682
I20260811 11:55:23.789959 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance
I20260811 11:55:23.790338 6718 external_mini_cluster.cc:1428] Setting key 3da30392431ff5e5f2b03dfac7bed797
Aug 11 11:55:23 dist-test-slave-zkpd krb5kdc[7280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449323, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:23.800997 7369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:23.788588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:47977 (local address 127.6.143.190:35819)
0811 11:55:23.788753 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:23.788758 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:23.789561 (+ 803us) server_negotiation.cc:407] Connection header received
0811 11:55:23.791331 (+ 1770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:23.791334 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:23.791395 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:23.791463 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:23.792922 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.793446 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.794587 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.794846 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.794930 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:23.797548 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:23.797552 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:23.797554 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:23.797579 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:23.799092 (+ 1513us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:23.799585 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:23.799587 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:23.799588 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:23.799667 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:23.800017 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:23.800020 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:23.800021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:23.800204 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:23.800281 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:23.800758 (+ 477us) server_negotiation.cc:299] Negotiation successful
0811 11:55:23.800863 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 11:55:23.801791 7820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:23.788968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:35819 (local address 127.6.143.131:47977)
0811 11:55:23.789410 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:23.789451 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:23.791091 (+ 1640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:23.791678 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:23.791689 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:23.792161 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:23.792790 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:23.792806 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.793549 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.793551 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:23.794445 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:23.794454 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.794983 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.794986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:23.795125 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:23.795763 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:23.795797 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:23.797377 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:23.799200 (+ 1823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:23.799205 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:23.799219 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:23.799493 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:23.799749 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:23.799751 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:23.799752 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:23.799934 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:23.800293 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:23.800298 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:23.800540 (+ 242us) client_negotiation.cc:769] Sending connection context
0811 11:55:23.800693 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 11:55:23.800953 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":114,"threads_started":1}
I20260811 11:55:23.803006 7818 heartbeater.cc:347] Connected to a master server at 127.6.143.190:35819
I20260811 11:55:23.803296 7818 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:23.803968 7818 heartbeater.cc:511] Master 127.6.143.190:35819 requested a full tablet report, sending...
I20260811 11:55:23.805070 7326 ts_manager.cc:194] Registered new tserver with Master: b02b4c7a95c54631a193dcf7551a9a98 (127.6.143.131:34659)
I20260811 11:55:23.805635 7326 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.131@KRBTEST.COM'} at 127.6.143.131:47977
I20260811 11:55:23.816082 6718 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:55:23.826257 7369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:23.819447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53532 (local address 127.6.143.190:35819)
0811 11:55:23.819620 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:23.819624 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:23.819724 (+ 100us) server_negotiation.cc:407] Connection header received
0811 11:55:23.819830 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:23.819832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:23.819877 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:23.819997 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:23.820663 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.821168 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.821948 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.822151 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.822223 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:23.823469 (+ 1246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:23.823473 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:23.823475 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:23.823498 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:23.824779 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:23.825153 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:23.825155 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:23.825157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:23.825230 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:23.825464 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:23.825466 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:23.825467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:23.825664 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:23.825763 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:23.825958 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 11:55:23.826047 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 11:55:23.830984 7326 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53532:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:55:23.834002 7326 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:55:23.848815 7836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:23.843071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:34659 (local address 127.0.0.1:32834)
0811 11:55:23.844080 (+ 1009us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:23.844095 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:23.844179 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:23.844813 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:23.844816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:23.844835 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:23.844928 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:23.844935 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.846229 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.846232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:23.847124 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:23.847135 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.848252 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.848261 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:23.848372 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:23.848411 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 11:55:23.848525 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 11:55:23.848649 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":925,"thread_start_us":87,"threads_started":1}
I20260811 11:55:23.848968 7833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:23.842334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47984 (local address 127.6.143.129:43541)
0811 11:55:23.843678 (+ 1344us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:23.843686 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:23.843712 (+ 26us) server_negotiation.cc:407] Connection header received
0811 11:55:23.843792 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:23.843796 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:23.844017 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:23.844412 (+ 395us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:23.844611 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.845777 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.847525 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.848324 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.848437 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:23.848569 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:23.848681 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 11:55:23.848773 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1232,"thread_start_us":114,"threads_started":1}
I20260811 11:55:23.849438 7837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:23.843166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48434 (local address 127.6.143.130:43177)
0811 11:55:23.843641 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:23.843651 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:23.843671 (+ 20us) server_negotiation.cc:407] Connection header received
0811 11:55:23.843734 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:23.843738 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:23.843970 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:23.844147 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:23.844607 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.845961 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.847569 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.848222 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.848333 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:23.848490 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:23.849186 (+ 696us) server_negotiation.cc:299] Negotiation successful
0811 11:55:23.849287 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":131,"threads_started":1}
I20260811 11:55:23.849082 7830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:23.842308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:43177 (local address 127.0.0.1:48434)
0811 11:55:23.843380 (+ 1072us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:23.843394 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:23.843488 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:23.844152 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:23.844161 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:23.844191 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:23.844418 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:23.844468 (+ 50us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.846151 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.846154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:23.847390 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:23.847403 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.848594 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.848600 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:23.848688 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:23.848709 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 11:55:23.848873 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 11:55:23.848966 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":996,"mutex_wait_us":77,"spinlock_wait_cycles":23168,"thread_start_us":107,"threads_started":1}
I20260811 11:55:23.849439 7832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:23.842189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:43541 (local address 127.0.0.1:47984)
0811 11:55:23.842683 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:23.842719 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:23.842859 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:23.844189 (+ 1330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:23.844191 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:23.844204 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:23.844461 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:23.844473 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.845901 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.845904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:23.847393 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:23.847407 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.848448 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.848452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:23.848542 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:23.848565 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 11:55:23.848649 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 11:55:23.848722 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":181,"thread_start_us":110,"threads_started":1}
I20260811 11:55:23.850037 7838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:23.843228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32834 (local address 127.6.143.131:34659)
0811 11:55:23.843683 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:23.843694 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:23.844279 (+ 585us) server_negotiation.cc:407] Connection header received
0811 11:55:23.844383 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:23.844388 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:23.844625 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:23.844837 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:23.845054 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.846082 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.847261 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.848067 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.848182 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:23.848330 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:23.849257 (+ 927us) server_negotiation.cc:299] Negotiation successful
0811 11:55:23.849399 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":124,"threads_started":1}
I20260811 11:55:23.852522 7462 tablet_service.cc:1467] Processing CreateTablet for tablet 9a7f71be1e1c43df908b0103f1ef4de7 (DEFAULT_TABLE table=test-table [id=b4a47d716ee547f19f43a1918b443c1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:23.852547 7750 tablet_service.cc:1467] Processing CreateTablet for tablet 9a7f71be1e1c43df908b0103f1ef4de7 (DEFAULT_TABLE table=test-table [id=b4a47d716ee547f19f43a1918b443c1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:23.853534 7462 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a7f71be1e1c43df908b0103f1ef4de7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:23.853615 7750 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a7f71be1e1c43df908b0103f1ef4de7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:23.853806 7609 tablet_service.cc:1467] Processing CreateTablet for tablet 9a7f71be1e1c43df908b0103f1ef4de7 (DEFAULT_TABLE table=test-table [id=b4a47d716ee547f19f43a1918b443c1e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:23.854737 7609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9a7f71be1e1c43df908b0103f1ef4de7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:23.859467 7839 tablet_bootstrap.cc:492] T 9a7f71be1e1c43df908b0103f1ef4de7 P a4f782977957425f99d5cfb954de2ac6: Bootstrap starting.
I20260811 11:55:23.861814 7839 tablet_bootstrap.cc:654] T 9a7f71be1e1c43df908b0103f1ef4de7 P a4f782977957425f99d5cfb954de2ac6: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:23.862453 7842 tablet_bootstrap.cc:492] T 9a7f71be1e1c43df908b0103f1ef4de7 P 3b97d16c560c408fb89383dfdd55987a: Bootstrap starting.
I20260811 11:55:23.862645 7839 log.cc:824] T 9a7f71be1e1c43df908b0103f1ef4de7 P a4f782977957425f99d5cfb954de2ac6: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:23.863024 7841 tablet_bootstrap.cc:492] T 9a7f71be1e1c43df908b0103f1ef4de7 P b02b4c7a95c54631a193dcf7551a9a98: Bootstrap starting.
I20260811 11:55:23.864558 7839 tablet_bootstrap.cc:492] T 9a7f71be1e1c43df908b0103f1ef4de7 P a4f782977957425f99d5cfb954de2ac6: No bootstrap required, opened a new log
I20260811 11:55:23.864775 7839 ts_tablet_manager.cc:1397] T 9a7f71be1e1c43df908b0103f1ef4de7 P a4f782977957425f99d5cfb954de2ac6: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 11:55:23.865211 7842 tablet_bootstrap.cc:654] T 9a7f71be1e1c43df908b0103f1ef4de7 P 3b97d16c560c408fb89383dfdd55987a: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:23.865669 7841 tablet_bootstrap.cc:654] T 9a7f71be1e1c43df908b0103f1ef4de7 P b02b4c7a95c54631a193dcf7551a9a98: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:23.866164 7842 log.cc:824] T 9a7f71be1e1c43df908b0103f1ef4de7 P 3b97d16c560c408fb89383dfdd55987a: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:23.866469 7841 log.cc:824] T 9a7f71be1e1c43df908b0103f1ef4de7 P b02b4c7a95c54631a193dcf7551a9a98: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:23.868129 7842 tablet_bootstrap.cc:492] T 9a7f71be1e1c43df908b0103f1ef4de7 P 3b97d16c560c408fb89383dfdd55987a: No bootstrap required, opened a new log
I20260811 11:55:23.868321 7842 ts_tablet_manager.cc:1397] T 9a7f71be1e1c43df908b0103f1ef4de7 P 3b97d16c560c408fb89383dfdd55987a: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260811 11:55:23.868877 7839 raft_consensus.cc:348] T 9a7f71be1e1c43df908b0103f1ef4de7 P a4f782977957425f99d5cfb954de2ac6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f782977957425f99d5cfb954de2ac6" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 43541 } } peers { permanent_uuid: "3b97d16c560c408fb89383dfdd55987a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43177 } } peers { permanent_uuid: "b02b4c7a95c54631a193dcf7551a9a98" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 34659 } }
I20260811 11:55:23.869161 7839 raft_consensus.cc:374] T 9a7f71be1e1c43df908b0103f1ef4de7 P a4f782977957425f99d5cfb954de2ac6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:23.869230 7839 raft_consensus.cc:729] T 9a7f71be1e1c43df908b0103f1ef4de7 P a4f782977957425f99d5cfb954de2ac6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4f782977957425f99d5cfb954de2ac6, State: Initialized, Role: FOLLOWER
I20260811 11:55:23.869531 7841 tablet_bootstrap.cc:492] T 9a7f71be1e1c43df908b0103f1ef4de7 P b02b4c7a95c54631a193dcf7551a9a98: No bootstrap required, opened a new log
I20260811 11:55:23.869709 7841 ts_tablet_manager.cc:1397] T 9a7f71be1e1c43df908b0103f1ef4de7 P b02b4c7a95c54631a193dcf7551a9a98: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 11:55:23.869740 7839 consensus_queue.cc:260] T 9a7f71be1e1c43df908b0103f1ef4de7 P a4f782977957425f99d5cfb954de2ac6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f782977957425f99d5cfb954de2ac6" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 43541 } } peers { permanent_uuid: "3b97d16c560c408fb89383dfdd55987a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43177 } } peers { permanent_uuid: "b02b4c7a95c54631a193dcf7551a9a98" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 34659 } }
I20260811 11:55:23.870437 7535 heartbeater.cc:503] Master 127.6.143.190:35819 was elected leader, sending a full tablet report...
I20260811 11:55:23.870779 7839 ts_tablet_manager.cc:1428] T 9a7f71be1e1c43df908b0103f1ef4de7 P a4f782977957425f99d5cfb954de2ac6: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 11:55:23.871958 7842 raft_consensus.cc:348] T 9a7f71be1e1c43df908b0103f1ef4de7 P 3b97d16c560c408fb89383dfdd55987a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f782977957425f99d5cfb954de2ac6" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 43541 } } peers { permanent_uuid: "3b97d16c560c408fb89383dfdd55987a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43177 } } peers { permanent_uuid: "b02b4c7a95c54631a193dcf7551a9a98" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 34659 } }
I20260811 11:55:23.872215 7842 raft_consensus.cc:374] T 9a7f71be1e1c43df908b0103f1ef4de7 P 3b97d16c560c408fb89383dfdd55987a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:23.872277 7842 raft_consensus.cc:729] T 9a7f71be1e1c43df908b0103f1ef4de7 P 3b97d16c560c408fb89383dfdd55987a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b97d16c560c408fb89383dfdd55987a, State: Initialized, Role: FOLLOWER
I20260811 11:55:23.872706 7842 consensus_queue.cc:260] T 9a7f71be1e1c43df908b0103f1ef4de7 P 3b97d16c560c408fb89383dfdd55987a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f782977957425f99d5cfb954de2ac6" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 43541 } } peers { permanent_uuid: "3b97d16c560c408fb89383dfdd55987a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43177 } } peers { permanent_uuid: "b02b4c7a95c54631a193dcf7551a9a98" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 34659 } }
I20260811 11:55:23.873340 7677 heartbeater.cc:503] Master 127.6.143.190:35819 was elected leader, sending a full tablet report...
I20260811 11:55:23.873699 7842 ts_tablet_manager.cc:1428] T 9a7f71be1e1c43df908b0103f1ef4de7 P 3b97d16c560c408fb89383dfdd55987a: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 11:55:23.874214 7841 raft_consensus.cc:348] T 9a7f71be1e1c43df908b0103f1ef4de7 P b02b4c7a95c54631a193dcf7551a9a98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f782977957425f99d5cfb954de2ac6" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 43541 } } peers { permanent_uuid: "3b97d16c560c408fb89383dfdd55987a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43177 } } peers { permanent_uuid: "b02b4c7a95c54631a193dcf7551a9a98" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 34659 } }
I20260811 11:55:23.874414 7841 raft_consensus.cc:374] T 9a7f71be1e1c43df908b0103f1ef4de7 P b02b4c7a95c54631a193dcf7551a9a98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:23.874487 7841 raft_consensus.cc:729] T 9a7f71be1e1c43df908b0103f1ef4de7 P b02b4c7a95c54631a193dcf7551a9a98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b02b4c7a95c54631a193dcf7551a9a98, State: Initialized, Role: FOLLOWER
I20260811 11:55:23.874919 7841 consensus_queue.cc:260] T 9a7f71be1e1c43df908b0103f1ef4de7 P b02b4c7a95c54631a193dcf7551a9a98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f782977957425f99d5cfb954de2ac6" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 43541 } } peers { permanent_uuid: "3b97d16c560c408fb89383dfdd55987a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43177 } } peers { permanent_uuid: "b02b4c7a95c54631a193dcf7551a9a98" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 34659 } }
I20260811 11:55:23.875756 7818 heartbeater.cc:503] Master 127.6.143.190:35819 was elected leader, sending a full tablet report...
I20260811 11:55:23.876004 7841 ts_tablet_manager.cc:1428] T 9a7f71be1e1c43df908b0103f1ef4de7 P b02b4c7a95c54631a193dcf7551a9a98: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260811 11:55:23.988524 7846 raft_consensus.cc:482] T 9a7f71be1e1c43df908b0103f1ef4de7 P a4f782977957425f99d5cfb954de2ac6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:55:23.988757 7846 raft_consensus.cc:504] T 9a7f71be1e1c43df908b0103f1ef4de7 P a4f782977957425f99d5cfb954de2ac6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f782977957425f99d5cfb954de2ac6" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 43541 } } peers { permanent_uuid: "3b97d16c560c408fb89383dfdd55987a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43177 } } peers { permanent_uuid: "b02b4c7a95c54631a193dcf7551a9a98" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 34659 } }
I20260811 11:55:23.989894 7846 leader_election.cc:288] T 9a7f71be1e1c43df908b0103f1ef4de7 P a4f782977957425f99d5cfb954de2ac6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3b97d16c560c408fb89383dfdd55987a (127.6.143.130:43177), b02b4c7a95c54631a193dcf7551a9a98 (127.6.143.131:34659)
I20260811 11:55:23.995622 7837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:23.990318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:60093 (local address 127.6.143.130:43177)
0811 11:55:23.990481 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:23.990486 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:23.990795 (+ 309us) server_negotiation.cc:407] Connection header received
0811 11:55:23.990983 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:23.990986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:23.991042 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:23.991142 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:23.991552 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.992382 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.994326 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.995052 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.995179 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:23.995231 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:23.995421 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 11:55:23.995490 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 11:55:23.995654 7849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:23.990192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:43177 (local address 127.6.143.129:60093)
0811 11:55:23.990671 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:23.990687 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:23.990838 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:23.991173 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:23.991176 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:23.991203 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:23.991427 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:23.991435 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.992513 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.992517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:23.994119 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:23.994133 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.995172 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.995175 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:23.995253 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:23.995275 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 11:55:23.995390 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 11:55:23.995466 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"mutex_wait_us":212,"spinlock_wait_cycles":39552,"thread_start_us":156,"threads_started":1}
I20260811 11:55:23.996225 7838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:23.990315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:48183 (local address 127.6.143.131:34659)
0811 11:55:23.990482 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:23.990487 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:23.990828 (+ 341us) server_negotiation.cc:407] Connection header received
0811 11:55:23.990983 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:23.990986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:23.991042 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:23.991141 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:23.991552 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.992383 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.994967 (+ 2584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.995578 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.995701 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:23.995750 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:23.996038 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 11:55:23.996115 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 11:55:23.996235 7850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:23.990192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:34659 (local address 127.6.143.129:48183)
0811 11:55:23.990671 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:23.990687 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:23.990838 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:23.991173 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:23.991176 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:23.991203 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:23.991427 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:23.991435 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.992513 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.992517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:23.994123 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:23.994133 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:23.995753 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:23.995759 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:23.995858 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:23.995881 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 11:55:23.996009 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 11:55:23.996078 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":396,"mutex_wait_us":224,"spinlock_wait_cycles":12544,"thread_start_us":201,"threads_started":1}
I20260811 11:55:23.996467 7629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a7f71be1e1c43df908b0103f1ef4de7" candidate_uuid: "a4f782977957425f99d5cfb954de2ac6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b97d16c560c408fb89383dfdd55987a" is_pre_election: true
I20260811 11:55:23.996831 7629 raft_consensus.cc:2454] T 9a7f71be1e1c43df908b0103f1ef4de7 P 3b97d16c560c408fb89383dfdd55987a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4f782977957425f99d5cfb954de2ac6 in term 0.
I20260811 11:55:23.996928 7770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a7f71be1e1c43df908b0103f1ef4de7" candidate_uuid: "a4f782977957425f99d5cfb954de2ac6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b02b4c7a95c54631a193dcf7551a9a98" is_pre_election: true
I20260811 11:55:23.997255 7770 raft_consensus.cc:2454] T 9a7f71be1e1c43df908b0103f1ef4de7 P b02b4c7a95c54631a193dcf7551a9a98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4f782977957425f99d5cfb954de2ac6 in term 0.
I20260811 11:55:23.997359 7416 leader_election.cc:302] T 9a7f71be1e1c43df908b0103f1ef4de7 P a4f782977957425f99d5cfb954de2ac6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b97d16c560c408fb89383dfdd55987a, a4f782977957425f99d5cfb954de2ac6; no voters:
I20260811 11:55:23.997684 7846 raft_consensus.cc:2781] T 9a7f71be1e1c43df908b0103f1ef4de7 P a4f782977957425f99d5cfb954de2ac6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:55:23.997782 7846 raft_consensus.cc:482] T 9a7f71be1e1c43df908b0103f1ef4de7 P a4f782977957425f99d5cfb954de2ac6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:55:23.997826 7846 raft_consensus.cc:3037] T 9a7f71be1e1c43df908b0103f1ef4de7 P a4f782977957425f99d5cfb954de2ac6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:23.999133 7846 raft_consensus.cc:504] T 9a7f71be1e1c43df908b0103f1ef4de7 P a4f782977957425f99d5cfb954de2ac6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f782977957425f99d5cfb954de2ac6" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 43541 } } peers { permanent_uuid: "3b97d16c560c408fb89383dfdd55987a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43177 } } peers { permanent_uuid: "b02b4c7a95c54631a193dcf7551a9a98" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 34659 } }
I20260811 11:55:23.999507 7846 leader_election.cc:288] T 9a7f71be1e1c43df908b0103f1ef4de7 P a4f782977957425f99d5cfb954de2ac6 [CANDIDATE]: Term 1 election: Requested vote from peers 3b97d16c560c408fb89383dfdd55987a (127.6.143.130:43177), b02b4c7a95c54631a193dcf7551a9a98 (127.6.143.131:34659)
I20260811 11:55:23.999862 7770 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a7f71be1e1c43df908b0103f1ef4de7" candidate_uuid: "a4f782977957425f99d5cfb954de2ac6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b02b4c7a95c54631a193dcf7551a9a98"
I20260811 11:55:23.999863 7629 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9a7f71be1e1c43df908b0103f1ef4de7" candidate_uuid: "a4f782977957425f99d5cfb954de2ac6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b97d16c560c408fb89383dfdd55987a"
I20260811 11:55:24.000020 7770 raft_consensus.cc:3037] T 9a7f71be1e1c43df908b0103f1ef4de7 P b02b4c7a95c54631a193dcf7551a9a98 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:24.000025 7629 raft_consensus.cc:3037] T 9a7f71be1e1c43df908b0103f1ef4de7 P 3b97d16c560c408fb89383dfdd55987a [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:24.000972 7770 raft_consensus.cc:2454] T 9a7f71be1e1c43df908b0103f1ef4de7 P b02b4c7a95c54631a193dcf7551a9a98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4f782977957425f99d5cfb954de2ac6 in term 1.
I20260811 11:55:24.001137 7629 raft_consensus.cc:2454] T 9a7f71be1e1c43df908b0103f1ef4de7 P 3b97d16c560c408fb89383dfdd55987a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4f782977957425f99d5cfb954de2ac6 in term 1.
I20260811 11:55:24.001299 7415 leader_election.cc:302] T 9a7f71be1e1c43df908b0103f1ef4de7 P a4f782977957425f99d5cfb954de2ac6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4f782977957425f99d5cfb954de2ac6, b02b4c7a95c54631a193dcf7551a9a98; no voters:
I20260811 11:55:24.001488 7846 raft_consensus.cc:2781] T 9a7f71be1e1c43df908b0103f1ef4de7 P a4f782977957425f99d5cfb954de2ac6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:24.001739 7846 raft_consensus.cc:686] T 9a7f71be1e1c43df908b0103f1ef4de7 P a4f782977957425f99d5cfb954de2ac6 [term 1 LEADER]: Becoming Leader. State: Replica: a4f782977957425f99d5cfb954de2ac6, State: Running, Role: LEADER
I20260811 11:55:24.002079 7846 consensus_queue.cc:237] T 9a7f71be1e1c43df908b0103f1ef4de7 P a4f782977957425f99d5cfb954de2ac6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f782977957425f99d5cfb954de2ac6" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 43541 } } peers { permanent_uuid: "3b97d16c560c408fb89383dfdd55987a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43177 } } peers { permanent_uuid: "b02b4c7a95c54631a193dcf7551a9a98" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 34659 } }
I20260811 11:55:24.005376 7326 catalog_manager.cc:5495] T 9a7f71be1e1c43df908b0103f1ef4de7 P a4f782977957425f99d5cfb954de2ac6 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4f782977957425f99d5cfb954de2ac6 (127.6.143.129). New cstate: current_term: 1 leader_uuid: "a4f782977957425f99d5cfb954de2ac6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a4f782977957425f99d5cfb954de2ac6" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 43541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3b97d16c560c408fb89383dfdd55987a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43177 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b02b4c7a95c54631a193dcf7551a9a98" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 34659 } health_report { overall_health: UNKNOWN } } }
W20260811 11:55:24.037730 7819 tablet.cc:2369] T 9a7f71be1e1c43df908b0103f1ef4de7 P b02b4c7a95c54631a193dcf7551a9a98: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 11:55:24 dist-test-slave-zkpd krb5kdc[7280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449322, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.129@KRBTEST.COM
I20260811 11:55:24.047079 7833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:24.037736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47992 (local address 127.6.143.129:43541)
0811 11:55:24.037918 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:24.037925 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:24.038118 (+ 193us) server_negotiation.cc:407] Connection header received
0811 11:55:24.038258 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:24.038261 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:24.038317 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:24.038395 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:24.039125 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:24.039681 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:24.040516 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:24.040756 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:24.040847 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:24.043442 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:24.043454 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:24.043470 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:24.043494 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:24.045188 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:24.045666 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:24.045673 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:24.045674 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:24.045755 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:24.046060 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:24.046063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:24.046065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:24.046313 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:24.046592 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:24.046731 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 11:55:24.046848 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 11:55:24.054862 7536 tablet.cc:2369] T 9a7f71be1e1c43df908b0103f1ef4de7 P a4f782977957425f99d5cfb954de2ac6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 11:55:24.055781 7678 tablet.cc:2369] T 9a7f71be1e1c43df908b0103f1ef4de7 P 3b97d16c560c408fb89383dfdd55987a: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 11:55:24 dist-test-slave-zkpd krb5kdc[7280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449324, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 11:55:24 dist-test-slave-zkpd krb5kdc[7280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449324, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.143.129@KRBTEST.COM
I20260811 11:55:24.080648 7833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:24.067343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48000 (local address 127.6.143.129:43541)
0811 11:55:24.067502 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:24.067508 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:24.067691 (+ 183us) server_negotiation.cc:407] Connection header received
0811 11:55:24.067871 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:24.067874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:24.067972 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:24.068060 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:24.068751 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:24.069298 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:24.070257 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:24.070512 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:24.070623 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:24.077617 (+ 6994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:24.077620 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:24.077622 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:24.077650 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:24.078988 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:24.079396 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:24.079399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:24.079400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:24.079477 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:24.079720 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:24.079722 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:24.079723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:24.080061 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:24.080178 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:24.080427 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 11:55:24.080527 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 11:55:24 dist-test-slave-zkpd krb5kdc[7280](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 11:55:24 dist-test-slave-zkpd krb5kdc[7280](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449324, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.129@KRBTEST.COM
I20260811 11:55:24.107405 7833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:24.099411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48008 (local address 127.6.143.129:43541)
0811 11:55:24.099597 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:24.099600 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:24.099667 (+ 67us) server_negotiation.cc:407] Connection header received
0811 11:55:24.099782 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:24.099785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:24.099827 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:24.099942 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:24.100620 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:24.101113 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:24.101968 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:24.102190 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:24.102300 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:24.104682 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:24.104685 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:24.104687 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:24.104711 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:24.105905 (+ 1194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:24.106309 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:24.106312 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:24.106313 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:24.106393 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:24.106624 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:24.106626 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:24.106627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:24.106777 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:24.106890 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:24.107089 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 11:55:24.107183 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260811 11:55:24.108856 7440 tablet_service.cc:2908] Rejecting scan request for tablet 9a7f71be1e1c43df908b0103f1ef4de7: Uninitialized: clean time has not yet been initialized
I20260811 11:55:24.110929 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 7394
I20260811 11:55:24.119633 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 7540
I20260811 11:55:24.126869 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 7682
I20260811 11:55:24.134248 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 7293
2026-08-11T11:55:24Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2514 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:55:24 dist-test-slave-zkpd krb5kdc[7865](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:55:24 dist-test-slave-zkpd krb5kdc[7865](info): set up 2 sockets
Aug 11 11:55:24 dist-test-slave-zkpd krb5kdc[7865](info): commencing operation
krb5kdc: starting...
W20260811 11:55:25.212386 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:55:25 dist-test-slave-zkpd krb5kdc[7865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:55:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:55:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
I20260811 11:55:25.337121 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:39261
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:46545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:39261
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:25.448750 7878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:25.448994 7878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:25.449069 7878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:25.453085 7878 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:55:25.453135 7878 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 11:55:25.453152 7878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:25.453197 7878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:25.453229 7878 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:55:25.453251 7878 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:55:25.458556 7878 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:46545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:39261
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:39261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:25.459556 7878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:25.460602 7878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:25.470249 7884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:25.470263 7883 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:25.470283 7887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:25.470805 7878 server_base.cc:1034] running on GCE node
I20260811 11:55:25.471339 7878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:25.472558 7878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:25.473727 7878 hybrid_clock.cc:648] HybridClock initialized: now 1786449325473705 us; error 31 us; skew 500 ppm
Aug 11 11:55:25 dist-test-slave-zkpd krb5kdc[7865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449325, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:25.476504 7878 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:55:25.477308 7878 webserver.cc:460] Webserver started at http://127.6.143.190:38677/ using document root <none> and password file <none>
I20260811 11:55:25.477830 7878 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:25.477890 7878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:25.478055 7878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:25.480082 7878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "64e8c570ad8a4963935c53b05a7e3bd6"
format_stamp: "Formatted at 2026-08-11 11:55:25 on dist-test-slave-zkpd"
server_key: "4b065eaa3ca7de85f13a5245c0954306"
server_key_iv: "816755feab5b78f16163816960747f46"
server_key_version: "encryptionkey@0"
I20260811 11:55:25.480595 7878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "64e8c570ad8a4963935c53b05a7e3bd6"
format_stamp: "Formatted at 2026-08-11 11:55:25 on dist-test-slave-zkpd"
server_key: "4b065eaa3ca7de85f13a5245c0954306"
server_key_iv: "816755feab5b78f16163816960747f46"
server_key_version: "encryptionkey@0"
I20260811 11:55:25.484169 7878 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 11:55:25.486442 7895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:25.487637 7878 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:25.487790 7878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "64e8c570ad8a4963935c53b05a7e3bd6"
format_stamp: "Formatted at 2026-08-11 11:55:25 on dist-test-slave-zkpd"
server_key: "4b065eaa3ca7de85f13a5245c0954306"
server_key_iv: "816755feab5b78f16163816960747f46"
server_key_version: "encryptionkey@0"
I20260811 11:55:25.487918 7878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:25.505452 7878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:25.508173 7878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:25.508410 7878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:25.516251 7878 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:39261
I20260811 11:55:25.516273 7957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:39261 every 8 connection(s)
I20260811 11:55:25.517136 7878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:55:25.519793 7958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:25.523319 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 7878
I20260811 11:55:25.523458 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:55:25.523726 6718 external_mini_cluster.cc:1428] Setting key 612c7480168df4afdb10786feabf692c
I20260811 11:55:25.525738 7958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64e8c570ad8a4963935c53b05a7e3bd6: Bootstrap starting.
I20260811 11:55:25.528254 7958 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64e8c570ad8a4963935c53b05a7e3bd6: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:25.528999 7958 log.cc:824] T 00000000000000000000000000000000 P 64e8c570ad8a4963935c53b05a7e3bd6: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:25.531085 7958 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64e8c570ad8a4963935c53b05a7e3bd6: No bootstrap required, opened a new log
Aug 11 11:55:25 dist-test-slave-zkpd krb5kdc[7865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449325, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:25.534606 7958 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64e8c570ad8a4963935c53b05a7e3bd6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e8c570ad8a4963935c53b05a7e3bd6" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39261 } }
I20260811 11:55:25.534839 7958 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64e8c570ad8a4963935c53b05a7e3bd6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:25.534936 7958 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64e8c570ad8a4963935c53b05a7e3bd6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64e8c570ad8a4963935c53b05a7e3bd6, State: Initialized, Role: FOLLOWER
I20260811 11:55:25.535403 7958 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64e8c570ad8a4963935c53b05a7e3bd6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e8c570ad8a4963935c53b05a7e3bd6" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39261 } }
I20260811 11:55:25.535555 7958 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64e8c570ad8a4963935c53b05a7e3bd6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:55:25.535642 7958 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64e8c570ad8a4963935c53b05a7e3bd6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:55:25.535739 7958 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64e8c570ad8a4963935c53b05a7e3bd6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:25.536594 7958 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64e8c570ad8a4963935c53b05a7e3bd6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e8c570ad8a4963935c53b05a7e3bd6" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39261 } }
I20260811 11:55:25.536959 7958 leader_election.cc:302] T 00000000000000000000000000000000 P 64e8c570ad8a4963935c53b05a7e3bd6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64e8c570ad8a4963935c53b05a7e3bd6; no voters:
I20260811 11:55:25.537328 7958 leader_election.cc:288] T 00000000000000000000000000000000 P 64e8c570ad8a4963935c53b05a7e3bd6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:55:25.537395 7963 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64e8c570ad8a4963935c53b05a7e3bd6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:25.537606 7963 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64e8c570ad8a4963935c53b05a7e3bd6 [term 1 LEADER]: Becoming Leader. State: Replica: 64e8c570ad8a4963935c53b05a7e3bd6, State: Running, Role: LEADER
I20260811 11:55:25.537887 7963 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64e8c570ad8a4963935c53b05a7e3bd6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e8c570ad8a4963935c53b05a7e3bd6" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39261 } }
I20260811 11:55:25.538137 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:25.525108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33966 (local address 127.6.143.190:39261)
0811 11:55:25.525509 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:25.525524 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:25.525559 (+ 35us) server_negotiation.cc:407] Connection header received
0811 11:55:25.526311 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:25.526331 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:25.526667 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:25.526932 (+ 265us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:25.527802 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:25.528903 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:25.529917 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:25.530366 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:25.530483 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:25.533196 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:25.533202 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:25.533223 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:25.533249 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:25.535098 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:25.535534 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:25.535543 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:25.535550 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:25.535715 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:25.536002 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:25.536006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:25.536008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:25.536378 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:25.536588 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:25.536869 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 11:55:25.537188 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":120,"threads_started":1}
I20260811 11:55:25.538560 7958 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64e8c570ad8a4963935c53b05a7e3bd6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:55:25.539466 7964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e8c570ad8a4963935c53b05a7e3bd6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64e8c570ad8a4963935c53b05a7e3bd6. Latest consensus state: current_term: 1 leader_uuid: "64e8c570ad8a4963935c53b05a7e3bd6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e8c570ad8a4963935c53b05a7e3bd6" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39261 } } }
I20260811 11:55:25.539608 7964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e8c570ad8a4963935c53b05a7e3bd6 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:25.539963 7965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64e8c570ad8a4963935c53b05a7e3bd6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64e8c570ad8a4963935c53b05a7e3bd6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64e8c570ad8a4963935c53b05a7e3bd6" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39261 } } }
I20260811 11:55:25.540053 7965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64e8c570ad8a4963935c53b05a7e3bd6 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:25.541199 7972 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:55:25.543936 7972 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:55:25.549230 7972 catalog_manager.cc:1347] Generated new cluster ID: 47dd4fdaeb3742c4ae8d7fd21a7337a3
I20260811 11:55:25.549290 7972 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:55:25.576217 7972 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:55:25.577466 7972 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:55:25.583518 7972 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64e8c570ad8a4963935c53b05a7e3bd6: Generated new TSK 0
I20260811 11:55:25.584292 7972 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
I20260811 11:55:25.613037 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:0
--local_ip_for_outbound_sockets=127.6.143.129
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:39261
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:46545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:25.724694 7987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:25.724943 7987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:25.724985 7987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:25.729030 7987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:25.729079 7987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:25.729117 7987 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 11:55:25.729148 7987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:55:25.734264 7987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:46545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.143.190:39261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7987
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:25.735256 7987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:25.736363 7987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:25.746148 7992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:25.746148 7995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:25.746150 7993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:25.746523 7987 server_base.cc:1034] running on GCE node
I20260811 11:55:25.746930 7987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:25.747530 7987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:25.748746 7987 hybrid_clock.cc:648] HybridClock initialized: now 1786449325748716 us; error 46 us; skew 500 ppm
Aug 11 11:55:25 dist-test-slave-zkpd krb5kdc[7865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449325, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:25.752172 7987 init.cc:377] Logged in from keytab as kudu/127.6.143.129@KRBTEST.COM (short username kudu)
I20260811 11:55:25.753043 7987 webserver.cc:460] Webserver started at http://127.6.143.129:40475/ using document root <none> and password file <none>
I20260811 11:55:25.753602 7987 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:25.753706 7987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:25.753939 7987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:25.755878 7987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-0/data/instance:
uuid: "19e6de45be384403b1cfa03055276e92"
format_stamp: "Formatted at 2026-08-11 11:55:25 on dist-test-slave-zkpd"
server_key: "823d39a7e7d0defc8268c164c3c11338"
server_key_iv: "1675fa7b2646c2d629dbe30f18625cf8"
server_key_version: "encryptionkey@0"
I20260811 11:55:25.756527 7987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance:
uuid: "19e6de45be384403b1cfa03055276e92"
format_stamp: "Formatted at 2026-08-11 11:55:25 on dist-test-slave-zkpd"
server_key: "823d39a7e7d0defc8268c164c3c11338"
server_key_iv: "1675fa7b2646c2d629dbe30f18625cf8"
server_key_version: "encryptionkey@0"
I20260811 11:55:25.760291 7987 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 11:55:25.762584 8002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:25.763727 7987 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:25.763859 7987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "19e6de45be384403b1cfa03055276e92"
format_stamp: "Formatted at 2026-08-11 11:55:25 on dist-test-slave-zkpd"
server_key: "823d39a7e7d0defc8268c164c3c11338"
server_key_iv: "1675fa7b2646c2d629dbe30f18625cf8"
server_key_version: "encryptionkey@0"
I20260811 11:55:25.764022 7987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:25.791754 7987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:25.795222 7987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:25.795444 7987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:25.797008 7987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:25.797086 7987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:25.797161 7987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:25.797212 7987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 11:55:25 dist-test-slave-zkpd krb5kdc[7865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449325, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:25.808630 7987 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:37275
I20260811 11:55:25.808717 8120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:37275 every 8 connection(s)
I20260811 11:55:25.809702 7987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:55:25.811180 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:25.797905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:42161 (local address 127.6.143.190:39261)
0811 11:55:25.798036 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:25.798039 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:25.799137 (+ 1098us) server_negotiation.cc:407] Connection header received
0811 11:55:25.800341 (+ 1204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:25.800345 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:25.800417 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:25.800530 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:25.802216 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:25.802829 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:25.804004 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:25.804291 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:25.804429 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:25.807223 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:25.807228 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:25.807233 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:25.807269 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:25.808993 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:25.809550 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:25.809553 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:25.809554 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:25.809628 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:25.810025 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:25.810028 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:25.810030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:25.810292 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:25.810400 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:25.810957 (+ 557us) server_negotiation.cc:299] Negotiation successful
0811 11:55:25.811062 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 11:55:25.812005 8011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:25.798464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39261 (local address 127.6.143.129:42161)
0811 11:55:25.798971 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:25.799016 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:25.800113 (+ 1097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:25.800777 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:25.800789 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:25.801286 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:25.802030 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:25.802056 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:25.802965 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:25.802968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:25.803819 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:25.803829 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:25.804493 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:25.804497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:25.804659 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:25.805252 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:25.805295 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:25.807056 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:25.809101 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:25.809107 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:25.809120 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:25.809450 (+ 330us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:25.809723 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:25.809727 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:25.809729 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:25.809890 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:25.810458 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:25.810464 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:25.810718 (+ 254us) client_negotiation.cc:769] Sending connection context
0811 11:55:25.810882 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 11:55:25.811114 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":142,"threads_started":1}
I20260811 11:55:25.813238 8121 heartbeater.cc:347] Connected to a master server at 127.6.143.190:39261
I20260811 11:55:25.813373 8121 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:25.813925 8121 heartbeater.cc:511] Master 127.6.143.190:39261 requested a full tablet report, sending...
I20260811 11:55:25.815618 7911 ts_manager.cc:194] Registered new tserver with Master: 19e6de45be384403b1cfa03055276e92 (127.6.143.129:37275)
I20260811 11:55:25.817067 7911 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.129@KRBTEST.COM'} at 127.6.143.129:42161
I20260811 11:55:25.819609 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 7987
I20260811 11:55:25.819730 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance
I20260811 11:55:25.820111 6718 external_mini_cluster.cc:1428] Setting key a817138dcdfaf4d6a842eb4ee9eb3912
I20260811 11:55:25.824620 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:25.815617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:55715 (local address 127.6.143.190:39261)
0811 11:55:25.815731 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:25.815738 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:25.815935 (+ 197us) server_negotiation.cc:407] Connection header received
0811 11:55:25.816014 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:25.816016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:25.816065 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:25.816125 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:25.816590 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:25.817105 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:25.818095 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:25.818294 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:25.818369 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:25.819451 (+ 1082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:25.819456 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:25.819459 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:25.819487 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:25.821012 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:25.821431 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:25.821433 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:25.821455 (+ 22us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:25.821547 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:25.823751 (+ 2204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:25.823754 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:25.823756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:25.823996 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:25.824100 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:25.824370 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 11:55:25.824475 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":47}
I20260811 11:55:25.824628 8127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:25.815490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39261 (local address 127.6.143.129:55715)
0811 11:55:25.815807 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:25.815822 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:25.815940 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:25.816167 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:25.816170 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:25.816368 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:25.816477 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:25.816483 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:25.817252 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:25.817256 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:25.817980 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:25.817991 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:25.818409 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:25.818413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:25.818498 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:25.818908 (+ 410us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:25.818922 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:25.819307 (+ 385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:25.821127 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:25.821130 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:25.821131 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:25.821319 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:25.823532 (+ 2213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:25.823535 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:25.823536 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:25.823615 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:25.824102 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:25.824105 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:25.824194 (+ 89us) client_negotiation.cc:769] Sending connection context
0811 11:55:25.824301 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 11:55:25.824438 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":111,"threads_started":1}
WARNING: no policy specified for kudu/127.6.143.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
I20260811 11:55:25.847062 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:0
--local_ip_for_outbound_sockets=127.6.143.130
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:39261
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:46545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:25.957367 8130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:25.957629 8130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:25.957706 8130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:25.961730 8130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:25.961810 8130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:25.961880 8130 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 11:55:25.961961 8130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:55:25.967052 8130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:46545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.143.190:39261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:25.968087 8130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:25.969054 8130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:25.978125 8135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:25.978132 8138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:25.978139 8136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:25.978396 8130 server_base.cc:1034] running on GCE node
I20260811 11:55:25.978758 8130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:25.979346 8130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:25.980506 8130 hybrid_clock.cc:648] HybridClock initialized: now 1786449325980487 us; error 30 us; skew 500 ppm
Aug 11 11:55:25 dist-test-slave-zkpd krb5kdc[7865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449325, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:25.983196 8130 init.cc:377] Logged in from keytab as kudu/127.6.143.130@KRBTEST.COM (short username kudu)
I20260811 11:55:25.984094 8130 webserver.cc:460] Webserver started at http://127.6.143.130:39611/ using document root <none> and password file <none>
I20260811 11:55:25.984663 8130 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:25.984724 8130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:25.984956 8130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:25.986882 8130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-1/data/instance:
uuid: "a141f65cdfe7445d80052c6d6976fa72"
format_stamp: "Formatted at 2026-08-11 11:55:25 on dist-test-slave-zkpd"
server_key: "91a381445fc2e3f5b7015afd1bdfe189"
server_key_iv: "cedfd06ef97adf124a95f955e8b3a8d2"
server_key_version: "encryptionkey@0"
I20260811 11:55:25.987429 8130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance:
uuid: "a141f65cdfe7445d80052c6d6976fa72"
format_stamp: "Formatted at 2026-08-11 11:55:25 on dist-test-slave-zkpd"
server_key: "91a381445fc2e3f5b7015afd1bdfe189"
server_key_iv: "cedfd06ef97adf124a95f955e8b3a8d2"
server_key_version: "encryptionkey@0"
I20260811 11:55:25.991014 8130 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260811 11:55:25.993538 8145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:25.994721 8130 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 11:55:25.994817 8130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "a141f65cdfe7445d80052c6d6976fa72"
format_stamp: "Formatted at 2026-08-11 11:55:25 on dist-test-slave-zkpd"
server_key: "91a381445fc2e3f5b7015afd1bdfe189"
server_key_iv: "cedfd06ef97adf124a95f955e8b3a8d2"
server_key_version: "encryptionkey@0"
I20260811 11:55:25.994951 8130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:26.014326 8130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:26.017591 8130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:26.017800 8130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:26.019243 8130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:26.019292 8130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:26.019353 8130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:26.019389 8130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 11:55:26 dist-test-slave-zkpd krb5kdc[7865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449325, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:26.030721 8130 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:36879
I20260811 11:55:26.030802 8268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:36879 every 8 connection(s)
I20260811 11:55:26.031767 8130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
I20260811 11:55:26.033617 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 8130
I20260811 11:55:26.033766 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance
I20260811 11:55:26.034047 6718 external_mini_cluster.cc:1428] Setting key bb89ab6e75e8c9df9d2b70d731f5cba3
I20260811 11:55:26.034245 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:26.020236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:34153 (local address 127.6.143.190:39261)
0811 11:55:26.020374 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:26.020378 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:26.021460 (+ 1082us) server_negotiation.cc:407] Connection header received
0811 11:55:26.022636 (+ 1176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:26.022640 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:26.022703 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:26.022824 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:26.024529 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.025398 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.027139 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.027524 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.027651 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:26.030409 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:26.030414 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:26.030417 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:26.030450 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:26.032091 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:26.032700 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:26.032704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:26.032706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:26.032790 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:26.033136 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:26.033140 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:26.033142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:26.033323 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:26.033453 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:26.033954 (+ 501us) server_negotiation.cc:299] Negotiation successful
0811 11:55:26.034075 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 11:55:26.035152 8155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:26.020756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39261 (local address 127.6.143.130:34153)
0811 11:55:26.021286 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:26.021328 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:26.022402 (+ 1074us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:26.023059 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:26.023072 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:26.023547 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:26.024314 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:26.024340 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.025536 (+ 1196us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.025540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:26.026984 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:26.026995 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.027689 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.027693 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:26.027870 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:26.028547 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:26.028570 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:26.030232 (+ 1662us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:26.032218 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:26.032225 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:26.032243 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:26.032589 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:26.032892 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:26.032895 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:26.032897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:26.033035 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:26.033437 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:26.033444 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:26.033692 (+ 248us) client_negotiation.cc:769] Sending connection context
0811 11:55:26.033901 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 11:55:26.034199 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":142,"threads_started":1}
I20260811 11:55:26.036419 8269 heartbeater.cc:347] Connected to a master server at 127.6.143.190:39261
I20260811 11:55:26.036571 8269 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:26.037124 8269 heartbeater.cc:511] Master 127.6.143.190:39261 requested a full tablet report, sending...
I20260811 11:55:26.039373 7911 ts_manager.cc:194] Registered new tserver with Master: a141f65cdfe7445d80052c6d6976fa72 (127.6.143.130:36879)
I20260811 11:55:26.040199 7911 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.130@KRBTEST.COM'} at 127.6.143.130:34153
WARNING: no policy specified for kudu/127.6.143.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.131@KRBTEST.COM" created.
I20260811 11:55:26.053064 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:26.039632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:43225 (local address 127.6.143.190:39261)
0811 11:55:26.039750 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:26.039757 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:26.045892 (+ 6135us) server_negotiation.cc:407] Connection header received
0811 11:55:26.046017 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:26.046022 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:26.046063 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:26.046159 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:26.046591 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.047200 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.048045 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.048326 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.048435 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:26.049544 (+ 1109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:26.049547 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:26.049550 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:26.049578 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:26.051181 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:26.051720 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:26.051723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:26.051724 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:26.051802 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:26.052141 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:26.052144 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:26.052145 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:26.052339 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:26.052429 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:26.052788 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 11:55:26.052943 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 11:55:26.053187 8276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:26.039325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39261 (local address 127.6.143.130:43225)
0811 11:55:26.045803 (+ 6478us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:26.045820 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:26.045937 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:26.046175 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:26.046178 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:26.046372 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:26.046472 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:26.046478 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.047326 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.047330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:26.047871 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:26.047879 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.048445 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.048448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:26.048540 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:26.049086 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:26.049098 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:26.049397 (+ 299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:26.051300 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:26.051306 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:26.051311 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:26.051599 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:26.051925 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:26.051931 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:26.051935 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:26.052042 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:26.052458 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:26.052464 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:26.052595 (+ 131us) client_negotiation.cc:769] Sending connection context
0811 11:55:26.052720 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 11:55:26.052890 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6364,"thread_start_us":106,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
I20260811 11:55:26.064340 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:0
--local_ip_for_outbound_sockets=127.6.143.131
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:39261
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:46545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:26.175005 8279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:26.175284 8279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:26.175364 8279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:26.179303 8279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:26.179388 8279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:26.179462 8279 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 11:55:26.179540 8279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:55:26.184597 8279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:46545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.143.190:39261
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:26.185561 8279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:26.186538 8279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:26.195926 8285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:26.195984 8279 server_base.cc:1034] running on GCE node
W20260811 11:55:26.196003 8284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:26.195931 8287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:26.196565 8279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:26.197198 8279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:26.198356 8279 hybrid_clock.cc:648] HybridClock initialized: now 1786449326198343 us; error 24 us; skew 500 ppm
Aug 11 11:55:26 dist-test-slave-zkpd krb5kdc[7865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449326, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:26.201107 8279 init.cc:377] Logged in from keytab as kudu/127.6.143.131@KRBTEST.COM (short username kudu)
I20260811 11:55:26.201978 8279 webserver.cc:460] Webserver started at http://127.6.143.131:35577/ using document root <none> and password file <none>
I20260811 11:55:26.202551 8279 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:26.202641 8279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:26.202853 8279 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:26.204896 8279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-2/data/instance:
uuid: "45df06e29b6248538b2691a04408c6b1"
format_stamp: "Formatted at 2026-08-11 11:55:26 on dist-test-slave-zkpd"
server_key: "b32913e305149c833b45349ce5584f14"
server_key_iv: "2cd1575cea7adac422567c8a7a88997a"
server_key_version: "encryptionkey@0"
I20260811 11:55:26.205469 8279 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance:
uuid: "45df06e29b6248538b2691a04408c6b1"
format_stamp: "Formatted at 2026-08-11 11:55:26 on dist-test-slave-zkpd"
server_key: "b32913e305149c833b45349ce5584f14"
server_key_iv: "2cd1575cea7adac422567c8a7a88997a"
server_key_version: "encryptionkey@0"
I20260811 11:55:26.209036 8279 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260811 11:55:26.211354 8294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:26.212577 8279 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:26.212721 8279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "45df06e29b6248538b2691a04408c6b1"
format_stamp: "Formatted at 2026-08-11 11:55:26 on dist-test-slave-zkpd"
server_key: "b32913e305149c833b45349ce5584f14"
server_key_iv: "2cd1575cea7adac422567c8a7a88997a"
server_key_version: "encryptionkey@0"
I20260811 11:55:26.212826 8279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:26.254069 8279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:26.257010 8279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:26.257213 8279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:26.258648 8279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:26.258721 8279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:26.258793 8279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:26.258837 8279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 11:55:26 dist-test-slave-zkpd krb5kdc[7865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449326, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:26.270092 8279 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:32903
I20260811 11:55:26.270169 8417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:32903 every 8 connection(s)
I20260811 11:55:26.270941 8279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:55:26.272634 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:26.259572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:58789 (local address 127.6.143.190:39261)
0811 11:55:26.259704 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:26.259708 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:26.260843 (+ 1135us) server_negotiation.cc:407] Connection header received
0811 11:55:26.261902 (+ 1059us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:26.261905 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:26.261966 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:26.262081 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:26.263768 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.264434 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.265634 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.265875 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.265991 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:26.268852 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:26.268856 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:26.268859 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:26.268893 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:26.270505 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:26.271104 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:26.271106 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:26.271107 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:26.271181 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:26.271603 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:26.271606 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:26.271608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:26.271832 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:26.271964 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:26.272424 (+ 460us) server_negotiation.cc:299] Negotiation successful
0811 11:55:26.272519 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 11:55:26.273420 8303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:26.260149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39261 (local address 127.6.143.131:58789)
0811 11:55:26.260679 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:26.260743 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:26.261670 (+ 927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:26.262297 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:26.262310 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:26.262866 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:26.263577 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:26.263600 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.264570 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.264579 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:26.265505 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:26.265516 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.266043 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.266048 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:26.266233 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:26.266844 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:26.266867 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:26.268688 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:26.270696 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:26.270701 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:26.270711 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:26.270953 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:26.271288 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:26.271294 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:26.271297 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:26.271487 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:26.271984 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:26.271991 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:26.272208 (+ 217us) client_negotiation.cc:769] Sending connection context
0811 11:55:26.272369 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 11:55:26.272606 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":148,"threads_started":1}
I20260811 11:55:26.274571 8418 heartbeater.cc:347] Connected to a master server at 127.6.143.190:39261
I20260811 11:55:26.274729 8418 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:26.275213 8418 heartbeater.cc:511] Master 127.6.143.190:39261 requested a full tablet report, sending...
I20260811 11:55:26.276296 7912 ts_manager.cc:194] Registered new tserver with Master: 45df06e29b6248538b2691a04408c6b1 (127.6.143.131:32903)
I20260811 11:55:26.276875 7912 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.131@KRBTEST.COM'} at 127.6.143.131:58789
I20260811 11:55:26.280980 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 8279
I20260811 11:55:26.281116 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance
I20260811 11:55:26.281447 6718 external_mini_cluster.cc:1428] Setting key 990339c92f3eb6a9116f1eb6cf72653e
I20260811 11:55:26.283448 6718 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:55:26.283516 8424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:26.276766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39261 (local address 127.6.143.131:46957)
0811 11:55:26.277060 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:26.277075 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:26.277174 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:26.277426 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:26.277429 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:26.277623 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:26.277711 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:26.277716 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.278447 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.278451 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:26.278990 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:26.278998 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.279455 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.279459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:26.279549 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:26.279997 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:26.280008 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:26.280277 (+ 269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:26.282055 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:26.282059 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:26.282061 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:26.282253 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:26.282540 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:26.282543 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:26.282545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:26.282658 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:26.283007 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:26.283011 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:26.283106 (+ 95us) client_negotiation.cc:769] Sending connection context
0811 11:55:26.283212 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 11:55:26.283346 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":104,"threads_started":1}
I20260811 11:55:26.283521 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:26.276806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:46957 (local address 127.6.143.190:39261)
0811 11:55:26.276919 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:26.276923 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:26.277159 (+ 236us) server_negotiation.cc:407] Connection header received
0811 11:55:26.277271 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:26.277274 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:26.277312 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:26.277405 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:26.277826 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.278310 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.279117 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.279342 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.279451 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:26.280411 (+ 960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:26.280415 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:26.280417 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:26.280440 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:26.281941 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:26.282355 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:26.282359 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:26.282362 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:26.282445 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:26.282741 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:26.282745 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:26.282747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:26.282907 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:26.283015 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:26.283282 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 11:55:26.283383 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 11:55:26 dist-test-slave-zkpd krb5kdc[7865](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449326, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 11:55:26 dist-test-slave-zkpd krb5kdc[7865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449326, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:26.310436 7961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:26.302376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33982 (local address 127.6.143.190:39261)
0811 11:55:26.302517 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:26.302521 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:26.302533 (+ 12us) server_negotiation.cc:407] Connection header received
0811 11:55:26.302573 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:26.302575 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:26.302614 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:26.302707 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:26.303422 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.303994 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.304824 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.305067 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.305207 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:26.307474 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:26.307477 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:26.307479 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:26.307503 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:26.308781 (+ 1278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:26.309201 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:26.309205 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:26.309207 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:26.309289 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:26.309581 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:26.309585 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:26.309586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:26.309760 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:26.309903 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:26.310150 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 11:55:26.310266 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 11:55:26.313243 7912 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33982:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:55:26.316737 7912 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:55:26.331444 8443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:26.326563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40880 (local address 127.6.143.129:37275)
0811 11:55:26.326838 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:26.326845 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:26.326864 (+ 19us) server_negotiation.cc:407] Connection header received
0811 11:55:26.326929 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:26.326933 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:26.327085 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:26.327217 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:26.327420 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.328346 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.329876 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.330659 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.330993 (+ 334us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:26.331149 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:26.331207 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 11:55:26.331303 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":88,"threads_started":1}
I20260811 11:55:26.331419 8438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:26.325097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:32903 (local address 127.0.0.1:51466)
0811 11:55:26.325923 (+ 826us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:26.325977 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:26.326119 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:26.327360 (+ 1241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:26.327362 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:26.327381 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:26.327494 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:26.327501 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.328885 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.328888 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:26.330009 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:26.330020 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.331046 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.331050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:26.331158 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:26.331177 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 11:55:26.331245 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 11:55:26.331295 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":86,"threads_started":1}
I20260811 11:55:26.331470 8439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:26.325207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51466 (local address 127.6.143.131:32903)
0811 11:55:26.326682 (+ 1475us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:26.326697 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:26.326720 (+ 23us) server_negotiation.cc:407] Connection header received
0811 11:55:26.326795 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:26.326800 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:26.327003 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:26.327210 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:26.327624 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.328754 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.330124 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.330867 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.331018 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:26.331174 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:26.331268 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 11:55:26.331346 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1367,"thread_start_us":129,"threads_started":1}
I20260811 11:55:26.331419 8442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:26.325948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:37275 (local address 127.0.0.1:40880)
0811 11:55:26.326200 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:26.326213 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:26.326295 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:26.327210 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:26.327212 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:26.327231 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:26.327308 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:26.327314 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.328467 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.328472 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:26.329751 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:26.329761 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.330806 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.330810 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:26.330972 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:26.331029 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 11:55:26.331133 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 11:55:26.331263 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":72,"thread_start_us":82,"threads_started":1}
I20260811 11:55:26.332386 8440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:26.325257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:36879 (local address 127.0.0.1:54448)
0811 11:55:26.326051 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:26.326065 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:26.326166 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:26.326875 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:26.326882 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:26.326912 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:26.327015 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:26.327020 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.328489 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.328493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:26.329926 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:26.329937 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.330759 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.330762 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:26.330880 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:26.331458 (+ 578us) client_negotiation.cc:769] Sending connection context
0811 11:55:26.332172 (+ 714us) client_negotiation.cc:240] Negotiation successful
0811 11:55:26.332247 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"mutex_wait_us":119,"thread_start_us":88,"threads_started":1}
I20260811 11:55:26.332876 8441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:26.325432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54448 (local address 127.6.143.130:36879)
0811 11:55:26.325899 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:26.325914 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:26.326266 (+ 352us) server_negotiation.cc:407] Connection header received
0811 11:55:26.326353 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:26.326358 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:26.326576 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:26.326739 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:26.327137 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.328352 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.330074 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.330629 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.330758 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:26.330941 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:26.332640 (+ 1699us) server_negotiation.cc:299] Negotiation successful
0811 11:55:26.332737 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":119,"threads_started":1}
I20260811 11:55:26.334318 8348 tablet_service.cc:1467] Processing CreateTablet for tablet 21d7817386674822897e936b2a9fa82f (DEFAULT_TABLE table=test-table [id=a671746b62a14134b050d5333137f3b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:26.334331 8053 tablet_service.cc:1467] Processing CreateTablet for tablet 21d7817386674822897e936b2a9fa82f (DEFAULT_TABLE table=test-table [id=a671746b62a14134b050d5333137f3b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:26.335287 8053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21d7817386674822897e936b2a9fa82f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:26.335332 8348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21d7817386674822897e936b2a9fa82f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:26.337275 8200 tablet_service.cc:1467] Processing CreateTablet for tablet 21d7817386674822897e936b2a9fa82f (DEFAULT_TABLE table=test-table [id=a671746b62a14134b050d5333137f3b9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:26.338227 8200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 21d7817386674822897e936b2a9fa82f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:26.342582 8444 tablet_bootstrap.cc:492] T 21d7817386674822897e936b2a9fa82f P 45df06e29b6248538b2691a04408c6b1: Bootstrap starting.
I20260811 11:55:26.343729 8445 tablet_bootstrap.cc:492] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92: Bootstrap starting.
I20260811 11:55:26.344724 8444 tablet_bootstrap.cc:654] T 21d7817386674822897e936b2a9fa82f P 45df06e29b6248538b2691a04408c6b1: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:26.345522 8444 log.cc:824] T 21d7817386674822897e936b2a9fa82f P 45df06e29b6248538b2691a04408c6b1: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:26.345752 8446 tablet_bootstrap.cc:492] T 21d7817386674822897e936b2a9fa82f P a141f65cdfe7445d80052c6d6976fa72: Bootstrap starting.
I20260811 11:55:26.345919 8445 tablet_bootstrap.cc:654] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:26.346786 8445 log.cc:824] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:26.348112 8446 tablet_bootstrap.cc:654] T 21d7817386674822897e936b2a9fa82f P a141f65cdfe7445d80052c6d6976fa72: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:26.348278 8444 tablet_bootstrap.cc:492] T 21d7817386674822897e936b2a9fa82f P 45df06e29b6248538b2691a04408c6b1: No bootstrap required, opened a new log
I20260811 11:55:26.348495 8444 ts_tablet_manager.cc:1397] T 21d7817386674822897e936b2a9fa82f P 45df06e29b6248538b2691a04408c6b1: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 11:55:26.348784 8445 tablet_bootstrap.cc:492] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92: No bootstrap required, opened a new log
I20260811 11:55:26.348907 8446 log.cc:824] T 21d7817386674822897e936b2a9fa82f P a141f65cdfe7445d80052c6d6976fa72: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:26.348976 8445 ts_tablet_manager.cc:1397] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 11:55:26.350449 8446 tablet_bootstrap.cc:492] T 21d7817386674822897e936b2a9fa82f P a141f65cdfe7445d80052c6d6976fa72: No bootstrap required, opened a new log
I20260811 11:55:26.350638 8446 ts_tablet_manager.cc:1397] T 21d7817386674822897e936b2a9fa82f P a141f65cdfe7445d80052c6d6976fa72: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 11:55:26.352583 8444 raft_consensus.cc:348] T 21d7817386674822897e936b2a9fa82f P 45df06e29b6248538b2691a04408c6b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } }
I20260811 11:55:26.352870 8444 raft_consensus.cc:374] T 21d7817386674822897e936b2a9fa82f P 45df06e29b6248538b2691a04408c6b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:26.352934 8444 raft_consensus.cc:729] T 21d7817386674822897e936b2a9fa82f P 45df06e29b6248538b2691a04408c6b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45df06e29b6248538b2691a04408c6b1, State: Initialized, Role: FOLLOWER
I20260811 11:55:26.353385 8444 consensus_queue.cc:260] T 21d7817386674822897e936b2a9fa82f P 45df06e29b6248538b2691a04408c6b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } }
I20260811 11:55:26.353402 8445 raft_consensus.cc:348] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } }
I20260811 11:55:26.353624 8445 raft_consensus.cc:374] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:26.353718 8445 raft_consensus.cc:729] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19e6de45be384403b1cfa03055276e92, State: Initialized, Role: FOLLOWER
I20260811 11:55:26.353591 8446 raft_consensus.cc:348] T 21d7817386674822897e936b2a9fa82f P a141f65cdfe7445d80052c6d6976fa72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } }
I20260811 11:55:26.353853 8446 raft_consensus.cc:374] T 21d7817386674822897e936b2a9fa82f P a141f65cdfe7445d80052c6d6976fa72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:26.353917 8446 raft_consensus.cc:729] T 21d7817386674822897e936b2a9fa82f P a141f65cdfe7445d80052c6d6976fa72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a141f65cdfe7445d80052c6d6976fa72, State: Initialized, Role: FOLLOWER
I20260811 11:55:26.354199 8445 consensus_queue.cc:260] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } }
I20260811 11:55:26.354346 8446 consensus_queue.cc:260] T 21d7817386674822897e936b2a9fa82f P a141f65cdfe7445d80052c6d6976fa72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } }
I20260811 11:55:26.354458 8444 ts_tablet_manager.cc:1428] T 21d7817386674822897e936b2a9fa82f P 45df06e29b6248538b2691a04408c6b1: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 11:55:26.354789 8418 heartbeater.cc:503] Master 127.6.143.190:39261 was elected leader, sending a full tablet report...
I20260811 11:55:26.354882 8121 heartbeater.cc:503] Master 127.6.143.190:39261 was elected leader, sending a full tablet report...
I20260811 11:55:26.355232 8446 ts_tablet_manager.cc:1428] T 21d7817386674822897e936b2a9fa82f P a141f65cdfe7445d80052c6d6976fa72: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 11:55:26.355587 8269 heartbeater.cc:503] Master 127.6.143.190:39261 was elected leader, sending a full tablet report...
I20260811 11:55:26.356181 8445 ts_tablet_manager.cc:1428] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260811 11:55:26.521939 8419 tablet.cc:2369] T 21d7817386674822897e936b2a9fa82f P 45df06e29b6248538b2691a04408c6b1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 11:55:26.533004 8270 tablet.cc:2369] T 21d7817386674822897e936b2a9fa82f P a141f65cdfe7445d80052c6d6976fa72: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:55:26.545948 8451 raft_consensus.cc:482] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:55:26.546165 8451 raft_consensus.cc:504] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } }
I20260811 11:55:26.547346 8451 leader_election.cc:288] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45df06e29b6248538b2691a04408c6b1 (127.6.143.131:32903), a141f65cdfe7445d80052c6d6976fa72 (127.6.143.130:36879)
I20260811 11:55:26.551601 8441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:26.547834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:57815 (local address 127.6.143.130:36879)
0811 11:55:26.548149 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:26.548154 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:26.548398 (+ 244us) server_negotiation.cc:407] Connection header received
0811 11:55:26.548561 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:26.548564 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:26.548613 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:26.548695 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:26.548963 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.549458 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.550629 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.551023 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.551154 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:26.551207 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:26.551404 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 11:55:26.551470 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 11:55:26.551644 8439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:26.547744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:48213 (local address 127.6.143.131:32903)
0811 11:55:26.548045 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:26.548050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:26.548287 (+ 237us) server_negotiation.cc:407] Connection header received
0811 11:55:26.548478 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:26.548481 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:26.548532 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:26.548611 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:26.548896 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.549429 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.550634 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.551048 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.551179 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:26.551237 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:26.551449 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 11:55:26.551516 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260811 11:55:26.551689 8453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:26.547636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:36879 (local address 127.6.143.129:57815)
0811 11:55:26.548269 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:26.548281 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:26.548450 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:26.548708 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:26.548711 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:26.548726 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:26.548851 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:26.548856 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.549567 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.549571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:26.550498 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:26.550508 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.551147 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.551151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:26.551247 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:26.551278 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 11:55:26.551385 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 11:55:26.551455 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"mutex_wait_us":155,"thread_start_us":121,"threads_started":1}
I20260811 11:55:26.551865 8454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:26.547636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:32903 (local address 127.6.143.129:48213)
0811 11:55:26.548156 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:26.548169 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:26.548341 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:26.548638 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:26.548641 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:26.548658 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:26.548786 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:26.548792 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.549543 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.549547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:26.550504 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:26.550515 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.551173 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.551177 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:26.551295 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:26.551326 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 11:55:26.551430 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 11:55:26.551501 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"mutex_wait_us":72,"thread_start_us":118,"threads_started":1}
I20260811 11:55:26.552392 8219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d7817386674822897e936b2a9fa82f" candidate_uuid: "19e6de45be384403b1cfa03055276e92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a141f65cdfe7445d80052c6d6976fa72" is_pre_election: true
I20260811 11:55:26.552573 8370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d7817386674822897e936b2a9fa82f" candidate_uuid: "19e6de45be384403b1cfa03055276e92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45df06e29b6248538b2691a04408c6b1" is_pre_election: true
I20260811 11:55:26.552742 8219 raft_consensus.cc:2454] T 21d7817386674822897e936b2a9fa82f P a141f65cdfe7445d80052c6d6976fa72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19e6de45be384403b1cfa03055276e92 in term 0.
I20260811 11:55:26.552935 8370 raft_consensus.cc:2454] T 21d7817386674822897e936b2a9fa82f P 45df06e29b6248538b2691a04408c6b1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 19e6de45be384403b1cfa03055276e92 in term 0.
I20260811 11:55:26.553330 8006 leader_election.cc:302] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19e6de45be384403b1cfa03055276e92, a141f65cdfe7445d80052c6d6976fa72; no voters:
I20260811 11:55:26.553639 8451 raft_consensus.cc:2781] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:55:26.553738 8451 raft_consensus.cc:482] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:55:26.553785 8451 raft_consensus.cc:3037] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:26.554916 8451 raft_consensus.cc:504] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } }
I20260811 11:55:26.555277 8451 leader_election.cc:288] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92 [CANDIDATE]: Term 1 election: Requested vote from peers 45df06e29b6248538b2691a04408c6b1 (127.6.143.131:32903), a141f65cdfe7445d80052c6d6976fa72 (127.6.143.130:36879)
I20260811 11:55:26.555665 8219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d7817386674822897e936b2a9fa82f" candidate_uuid: "19e6de45be384403b1cfa03055276e92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a141f65cdfe7445d80052c6d6976fa72"
I20260811 11:55:26.555702 8370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "21d7817386674822897e936b2a9fa82f" candidate_uuid: "19e6de45be384403b1cfa03055276e92" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45df06e29b6248538b2691a04408c6b1"
I20260811 11:55:26.555778 8219 raft_consensus.cc:3037] T 21d7817386674822897e936b2a9fa82f P a141f65cdfe7445d80052c6d6976fa72 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:26.555828 8370 raft_consensus.cc:3037] T 21d7817386674822897e936b2a9fa82f P 45df06e29b6248538b2691a04408c6b1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:26.556849 8219 raft_consensus.cc:2454] T 21d7817386674822897e936b2a9fa82f P a141f65cdfe7445d80052c6d6976fa72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19e6de45be384403b1cfa03055276e92 in term 1.
I20260811 11:55:26.557150 8006 leader_election.cc:302] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19e6de45be384403b1cfa03055276e92, a141f65cdfe7445d80052c6d6976fa72; no voters:
I20260811 11:55:26.557335 8451 raft_consensus.cc:2781] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:26.557582 8451 raft_consensus.cc:686] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92 [term 1 LEADER]: Becoming Leader. State: Replica: 19e6de45be384403b1cfa03055276e92, State: Running, Role: LEADER
I20260811 11:55:26.557583 8370 raft_consensus.cc:2454] T 21d7817386674822897e936b2a9fa82f P 45df06e29b6248538b2691a04408c6b1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 19e6de45be384403b1cfa03055276e92 in term 1.
I20260811 11:55:26.557914 8451 consensus_queue.cc:237] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } }
W20260811 11:55:26.561028 8122 tablet.cc:2369] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:55:26.561345 7912 catalog_manager.cc:5495] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92 reported cstate change: term changed from 0 to 1, leader changed from <none> to 19e6de45be384403b1cfa03055276e92 (127.6.143.129). New cstate: current_term: 1 leader_uuid: "19e6de45be384403b1cfa03055276e92" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } health_report { overall_health: HEALTHY } } }
I20260811 11:55:26.625618 8443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:26.621781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40886 (local address 127.6.143.129:37275)
0811 11:55:26.621942 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:26.621947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:26.621963 (+ 16us) server_negotiation.cc:407] Connection header received
0811 11:55:26.622173 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:26.622176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:26.622234 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:26.622344 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:55:26.622632 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.623154 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.624046 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.624320 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.624437 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:26.624780 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:55:26.625183 (+ 403us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:55:26.625314 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:26.625406 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 11:55:26.625466 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 11:55:26.631477 8370 raft_consensus.cc:1261] T 21d7817386674822897e936b2a9fa82f P 45df06e29b6248538b2691a04408c6b1 [term 1 FOLLOWER]: Refusing update from remote peer 19e6de45be384403b1cfa03055276e92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:26.631516 8219 raft_consensus.cc:1261] T 21d7817386674822897e936b2a9fa82f P a141f65cdfe7445d80052c6d6976fa72 [term 1 FOLLOWER]: Refusing update from remote peer 19e6de45be384403b1cfa03055276e92: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:26.632200 8455 consensus_queue.cc:1035] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:26.632434 8451 consensus_queue.cc:1035] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:26.638648 8458 mvcc.cc:205] Tried to move back new op lower bound from 7317296441875013632 to 7317296441584041984. Current Snapshot: MvccSnapshot[applied={T|T < 7317296441875013632}]
I20260811 11:55:26.639513 8459 mvcc.cc:205] Tried to move back new op lower bound from 7317296441875013632 to 7317296441584041984. Current Snapshot: MvccSnapshot[applied={T|T < 7317296441875013632}]
I20260811 11:55:26.650884 7912 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:55:26.651042 7912 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:55:26.653453 7912 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 64e8c570ad8a4963935c53b05a7e3bd6: Sending DeleteTablet for 3 replicas of tablet 21d7817386674822897e936b2a9fa82f
I20260811 11:55:26.654277 8200 tablet_service.cc:1514] Processing DeleteTablet for tablet 21d7817386674822897e936b2a9fa82f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:26 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:54448
I20260811 11:55:26.654335 8053 tablet_service.cc:1514] Processing DeleteTablet for tablet 21d7817386674822897e936b2a9fa82f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:26 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:40880
I20260811 11:55:26.654311 8348 tablet_service.cc:1514] Processing DeleteTablet for tablet 21d7817386674822897e936b2a9fa82f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:26 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:51466
I20260811 11:55:26.654735 8468 tablet_replica.cc:330] T 21d7817386674822897e936b2a9fa82f P a141f65cdfe7445d80052c6d6976fa72: stopping tablet replica
I20260811 11:55:26.654770 8469 tablet_replica.cc:330] T 21d7817386674822897e936b2a9fa82f P 45df06e29b6248538b2691a04408c6b1: stopping tablet replica
I20260811 11:55:26.655000 8470 tablet_replica.cc:330] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92: stopping tablet replica
I20260811 11:55:26.655174 8468 raft_consensus.cc:2229] T 21d7817386674822897e936b2a9fa82f P a141f65cdfe7445d80052c6d6976fa72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:55:26.655284 8470 raft_consensus.cc:2229] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92 [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:55:26.655611 8468 raft_consensus.cc:2258] T 21d7817386674822897e936b2a9fa82f P a141f65cdfe7445d80052c6d6976fa72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:26.655668 8470 raft_consensus.cc:2258] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:26.655627 7912 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33982:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:55:26.656082 7912 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:55:26.657136 8470 ts_tablet_manager.cc:1905] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:26.657440 8468 ts_tablet_manager.cc:1905] T 21d7817386674822897e936b2a9fa82f P a141f65cdfe7445d80052c6d6976fa72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:26.659950 8470 ts_tablet_manager.cc:1918] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:55:26.660054 8470 log.cc:1196] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-0/wal/wals/21d7817386674822897e936b2a9fa82f
I20260811 11:55:26.660372 8470 ts_tablet_manager.cc:1939] T 21d7817386674822897e936b2a9fa82f P 19e6de45be384403b1cfa03055276e92: Deleting consensus metadata
I20260811 11:55:26.660512 8468 ts_tablet_manager.cc:1918] T 21d7817386674822897e936b2a9fa82f P a141f65cdfe7445d80052c6d6976fa72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:55:26.660686 8468 log.cc:1196] T 21d7817386674822897e936b2a9fa82f P a141f65cdfe7445d80052c6d6976fa72: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-1/wal/wals/21d7817386674822897e936b2a9fa82f
I20260811 11:55:26.661103 8468 ts_tablet_manager.cc:1939] T 21d7817386674822897e936b2a9fa82f P a141f65cdfe7445d80052c6d6976fa72: Deleting consensus metadata
I20260811 11:55:26.661832 7897 catalog_manager.cc:4841] TS 19e6de45be384403b1cfa03055276e92 (127.6.143.129:37275): tablet 21d7817386674822897e936b2a9fa82f (table test-table [id=a671746b62a14134b050d5333137f3b9]) successfully deleted
I20260811 11:55:26.661978 8200 tablet_service.cc:1467] Processing CreateTablet for tablet 3a4ab234b3544900a55086271f9ce2bd (DEFAULT_TABLE table=test-table [id=6a5571282d2248ab8e6e73cc166ba9d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:26.662232 8200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a4ab234b3544900a55086271f9ce2bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:26.662395 8348 tablet_service.cc:1467] Processing CreateTablet for tablet 3a4ab234b3544900a55086271f9ce2bd (DEFAULT_TABLE table=test-table [id=6a5571282d2248ab8e6e73cc166ba9d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:26.662685 8348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a4ab234b3544900a55086271f9ce2bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:26.664297 8446 tablet_bootstrap.cc:492] T 3a4ab234b3544900a55086271f9ce2bd P a141f65cdfe7445d80052c6d6976fa72: Bootstrap starting.
I20260811 11:55:26.664491 8053 tablet_service.cc:1467] Processing CreateTablet for tablet 3a4ab234b3544900a55086271f9ce2bd (DEFAULT_TABLE table=test-table [id=6a5571282d2248ab8e6e73cc166ba9d1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:26.664712 8469 raft_consensus.cc:2229] T 21d7817386674822897e936b2a9fa82f P 45df06e29b6248538b2691a04408c6b1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:55:26.664736 8053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3a4ab234b3544900a55086271f9ce2bd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:26.664834 8444 tablet_bootstrap.cc:492] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1: Bootstrap starting.
I20260811 11:55:26.665123 8469 raft_consensus.cc:2258] T 21d7817386674822897e936b2a9fa82f P 45df06e29b6248538b2691a04408c6b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:26.665237 8446 tablet_bootstrap.cc:654] T 3a4ab234b3544900a55086271f9ce2bd P a141f65cdfe7445d80052c6d6976fa72: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:26.665850 8444 tablet_bootstrap.cc:654] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:26.666632 8446 tablet_bootstrap.cc:492] T 3a4ab234b3544900a55086271f9ce2bd P a141f65cdfe7445d80052c6d6976fa72: No bootstrap required, opened a new log
I20260811 11:55:26.666730 8446 ts_tablet_manager.cc:1397] T 3a4ab234b3544900a55086271f9ce2bd P a141f65cdfe7445d80052c6d6976fa72: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:26.667007 8445 tablet_bootstrap.cc:492] T 3a4ab234b3544900a55086271f9ce2bd P 19e6de45be384403b1cfa03055276e92: Bootstrap starting.
I20260811 11:55:26.667222 8444 tablet_bootstrap.cc:492] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1: No bootstrap required, opened a new log
I20260811 11:55:26.667299 8444 ts_tablet_manager.cc:1397] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:26.667560 8446 raft_consensus.cc:348] T 3a4ab234b3544900a55086271f9ce2bd P a141f65cdfe7445d80052c6d6976fa72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } }
I20260811 11:55:26.667662 8446 raft_consensus.cc:374] T 3a4ab234b3544900a55086271f9ce2bd P a141f65cdfe7445d80052c6d6976fa72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:26.667701 8446 raft_consensus.cc:729] T 3a4ab234b3544900a55086271f9ce2bd P a141f65cdfe7445d80052c6d6976fa72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a141f65cdfe7445d80052c6d6976fa72, State: Initialized, Role: FOLLOWER
I20260811 11:55:26.667769 8444 raft_consensus.cc:348] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } }
I20260811 11:55:26.667824 8446 consensus_queue.cc:260] T 3a4ab234b3544900a55086271f9ce2bd P a141f65cdfe7445d80052c6d6976fa72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } }
I20260811 11:55:26.667984 8444 raft_consensus.cc:374] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:26.668007 8445 tablet_bootstrap.cc:654] T 3a4ab234b3544900a55086271f9ce2bd P 19e6de45be384403b1cfa03055276e92: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:26.668032 8444 raft_consensus.cc:729] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45df06e29b6248538b2691a04408c6b1, State: Initialized, Role: FOLLOWER
I20260811 11:55:26.668128 8446 ts_tablet_manager.cc:1428] T 3a4ab234b3544900a55086271f9ce2bd P a141f65cdfe7445d80052c6d6976fa72: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 11:55:26.668213 8444 consensus_queue.cc:260] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } }
I20260811 11:55:26.668672 8469 ts_tablet_manager.cc:1905] T 21d7817386674822897e936b2a9fa82f P 45df06e29b6248538b2691a04408c6b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:26.670424 8444 ts_tablet_manager.cc:1428] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260811 11:55:26.671075 8469 ts_tablet_manager.cc:1918] T 21d7817386674822897e936b2a9fa82f P 45df06e29b6248538b2691a04408c6b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:55:26.671169 8469 log.cc:1196] T 21d7817386674822897e936b2a9fa82f P 45df06e29b6248538b2691a04408c6b1: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-2/wal/wals/21d7817386674822897e936b2a9fa82f
I20260811 11:55:26.671454 8469 ts_tablet_manager.cc:1939] T 21d7817386674822897e936b2a9fa82f P 45df06e29b6248538b2691a04408c6b1: Deleting consensus metadata
I20260811 11:55:26.672158 7899 catalog_manager.cc:4841] TS a141f65cdfe7445d80052c6d6976fa72 (127.6.143.130:36879): tablet 21d7817386674822897e936b2a9fa82f (table test-table [id=a671746b62a14134b050d5333137f3b9]) successfully deleted
I20260811 11:55:26.672670 8445 tablet_bootstrap.cc:492] T 3a4ab234b3544900a55086271f9ce2bd P 19e6de45be384403b1cfa03055276e92: No bootstrap required, opened a new log
I20260811 11:55:26.672775 8445 ts_tablet_manager.cc:1397] T 3a4ab234b3544900a55086271f9ce2bd P 19e6de45be384403b1cfa03055276e92: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260811 11:55:26.673079 7897 catalog_manager.cc:4841] TS 45df06e29b6248538b2691a04408c6b1 (127.6.143.131:32903): tablet 21d7817386674822897e936b2a9fa82f (table test-table [id=a671746b62a14134b050d5333137f3b9]) successfully deleted
I20260811 11:55:26.673302 8445 raft_consensus.cc:348] T 3a4ab234b3544900a55086271f9ce2bd P 19e6de45be384403b1cfa03055276e92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } }
I20260811 11:55:26.673411 8445 raft_consensus.cc:374] T 3a4ab234b3544900a55086271f9ce2bd P 19e6de45be384403b1cfa03055276e92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:26.673460 8445 raft_consensus.cc:729] T 3a4ab234b3544900a55086271f9ce2bd P 19e6de45be384403b1cfa03055276e92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19e6de45be384403b1cfa03055276e92, State: Initialized, Role: FOLLOWER
I20260811 11:55:26.673573 8445 consensus_queue.cc:260] T 3a4ab234b3544900a55086271f9ce2bd P 19e6de45be384403b1cfa03055276e92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } }
I20260811 11:55:26.673869 8445 ts_tablet_manager.cc:1428] T 3a4ab234b3544900a55086271f9ce2bd P 19e6de45be384403b1cfa03055276e92: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 11:55:26.901873 8450 raft_consensus.cc:482] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:55:26.902025 8450 raft_consensus.cc:504] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } }
I20260811 11:55:26.903195 8450 leader_election.cc:288] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19e6de45be384403b1cfa03055276e92 (127.6.143.129:37275), a141f65cdfe7445d80052c6d6976fa72 (127.6.143.130:36879)
I20260811 11:55:26.907074 8443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:26.903543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:50137 (local address 127.6.143.129:37275)
0811 11:55:26.903684 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:26.903689 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:26.903884 (+ 195us) server_negotiation.cc:407] Connection header received
0811 11:55:26.904055 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:26.904058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:26.904108 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:26.904188 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:26.904484 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.905028 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.906087 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.906500 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.906627 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:26.906684 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:26.906862 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 11:55:26.906942 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 11:55:26.907090 8476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:26.903432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:37275 (local address 127.6.143.131:50137)
0811 11:55:26.903788 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:26.903800 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:26.903940 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:26.904202 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:26.904205 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:26.904220 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:26.904369 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:26.904374 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.905145 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.905148 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:26.905960 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:26.905968 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.906622 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.906626 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:26.906726 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:26.906749 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 11:55:26.906845 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 11:55:26.906915 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":94,"threads_started":1}
I20260811 11:55:26.907096 8441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:26.903532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:48189 (local address 127.6.143.130:36879)
0811 11:55:26.903666 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:26.903671 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:26.904020 (+ 349us) server_negotiation.cc:407] Connection header received
0811 11:55:26.904174 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:26.904177 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:26.904228 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:26.904310 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:26.904606 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.905110 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.906130 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.906525 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.906653 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:26.906706 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:26.906916 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 11:55:26.906980 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 11:55:26.907111 8475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:26.903432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:36879 (local address 127.6.143.131:48189)
0811 11:55:26.903876 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:26.903931 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:26.904058 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:26.904323 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:26.904326 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:26.904360 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:26.904467 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:26.904481 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.905229 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.905233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:26.906001 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:26.906009 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.906649 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.906652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:26.906764 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:26.906792 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 11:55:26.906892 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 11:55:26.906961 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"mutex_wait_us":28,"thread_start_us":81,"threads_started":1}
I20260811 11:55:26.907568 8074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a4ab234b3544900a55086271f9ce2bd" candidate_uuid: "45df06e29b6248538b2691a04408c6b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19e6de45be384403b1cfa03055276e92" is_pre_election: true
I20260811 11:55:26.907750 8219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a4ab234b3544900a55086271f9ce2bd" candidate_uuid: "45df06e29b6248538b2691a04408c6b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a141f65cdfe7445d80052c6d6976fa72" is_pre_election: true
I20260811 11:55:26.907845 8074 raft_consensus.cc:2454] T 3a4ab234b3544900a55086271f9ce2bd P 19e6de45be384403b1cfa03055276e92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45df06e29b6248538b2691a04408c6b1 in term 0.
I20260811 11:55:26.907913 8219 raft_consensus.cc:2454] T 3a4ab234b3544900a55086271f9ce2bd P a141f65cdfe7445d80052c6d6976fa72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45df06e29b6248538b2691a04408c6b1 in term 0.
I20260811 11:55:26.908289 8298 leader_election.cc:302] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45df06e29b6248538b2691a04408c6b1, a141f65cdfe7445d80052c6d6976fa72; no voters:
I20260811 11:55:26.908577 8450 raft_consensus.cc:2781] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:55:26.908641 8450 raft_consensus.cc:482] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:55:26.908681 8450 raft_consensus.cc:3037] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:26.909663 8450 raft_consensus.cc:504] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } }
I20260811 11:55:26.909981 8450 leader_election.cc:288] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1 [CANDIDATE]: Term 1 election: Requested vote from peers 19e6de45be384403b1cfa03055276e92 (127.6.143.129:37275), a141f65cdfe7445d80052c6d6976fa72 (127.6.143.130:36879)
I20260811 11:55:26.910284 8074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a4ab234b3544900a55086271f9ce2bd" candidate_uuid: "45df06e29b6248538b2691a04408c6b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19e6de45be384403b1cfa03055276e92"
I20260811 11:55:26.910352 8219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3a4ab234b3544900a55086271f9ce2bd" candidate_uuid: "45df06e29b6248538b2691a04408c6b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a141f65cdfe7445d80052c6d6976fa72"
I20260811 11:55:26.910408 8074 raft_consensus.cc:3037] T 3a4ab234b3544900a55086271f9ce2bd P 19e6de45be384403b1cfa03055276e92 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:26.910475 8219 raft_consensus.cc:3037] T 3a4ab234b3544900a55086271f9ce2bd P a141f65cdfe7445d80052c6d6976fa72 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:26.911600 8219 raft_consensus.cc:2454] T 3a4ab234b3544900a55086271f9ce2bd P a141f65cdfe7445d80052c6d6976fa72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45df06e29b6248538b2691a04408c6b1 in term 1.
I20260811 11:55:26.911600 8074 raft_consensus.cc:2454] T 3a4ab234b3544900a55086271f9ce2bd P 19e6de45be384403b1cfa03055276e92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45df06e29b6248538b2691a04408c6b1 in term 1.
I20260811 11:55:26.911867 8296 leader_election.cc:302] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19e6de45be384403b1cfa03055276e92, 45df06e29b6248538b2691a04408c6b1; no voters:
I20260811 11:55:26.912092 8450 raft_consensus.cc:2781] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:26.912285 8450 raft_consensus.cc:686] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1 [term 1 LEADER]: Becoming Leader. State: Replica: 45df06e29b6248538b2691a04408c6b1, State: Running, Role: LEADER
I20260811 11:55:26.912549 8450 consensus_queue.cc:237] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } }
I20260811 11:55:26.915058 7910 catalog_manager.cc:5495] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45df06e29b6248538b2691a04408c6b1 (127.6.143.131). New cstate: current_term: 1 leader_uuid: "45df06e29b6248538b2691a04408c6b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } health_report { overall_health: HEALTHY } } }
Aug 11 11:55:26 dist-test-slave-zkpd krb5kdc[7865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449325, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:26.994498 8487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:26.985618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33996 (local address 127.6.143.190:39261)
0811 11:55:26.985858 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:26.985868 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:26.985996 (+ 128us) server_negotiation.cc:407] Connection header received
0811 11:55:26.986173 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:26.986176 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:26.986214 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:26.986309 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:26.987034 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.987566 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.988623 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.988858 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.988968 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:26.991253 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:26.991257 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:26.991259 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:26.991289 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:26.992731 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:26.993214 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:26.993219 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:26.993221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:26.993304 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:26.993585 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:26.993590 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:26.993592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:26.993778 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:26.993887 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:26.994178 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 11:55:26.994338 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":83,"server-negotiator.queue_time_us":186,"thread_start_us":108,"threads_started":1}
I20260811 11:55:26.994496 8486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:26.985573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39261 (local address 127.0.0.1:33996)
0811 11:55:26.985888 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:26.985903 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:26.986058 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:26.986331 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:26.986334 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:26.986805 (+ 471us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:26.986908 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:26.986915 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.987678 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.987685 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:26.988462 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:26.988475 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:26.988983 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:26.988987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:26.989105 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:26.989635 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:26.989648 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:26.991116 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:26.992850 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:26.992862 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:26.992869 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:26.993109 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:26.993407 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:26.993410 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:26.993412 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:26.993495 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:26.993887 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:26.993896 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:26.994012 (+ 116us) client_negotiation.cc:769] Sending connection context
0811 11:55:26.994132 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 11:55:26.994286 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"mutex_wait_us":16,"thread_start_us":89,"threads_started":1}
I20260811 11:55:26.998383 7910 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:33996:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 11:55:26.999123 7910 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:55:27.004030 8200 tablet_service.cc:1467] Processing CreateTablet for tablet bf3091efeb30476087adf42e8f4299bb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ad127723b659435a9a2d4eb9e0bd16ae]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 11:55:27.004029 8053 tablet_service.cc:1467] Processing CreateTablet for tablet bf3091efeb30476087adf42e8f4299bb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ad127723b659435a9a2d4eb9e0bd16ae]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 11:55:27.004177 8348 tablet_service.cc:1467] Processing CreateTablet for tablet bf3091efeb30476087adf42e8f4299bb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ad127723b659435a9a2d4eb9e0bd16ae]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 11:55:27.004444 8200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf3091efeb30476087adf42e8f4299bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:27.004444 8053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf3091efeb30476087adf42e8f4299bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:27.004608 8348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf3091efeb30476087adf42e8f4299bb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:27.006983 8444 tablet_bootstrap.cc:492] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1: Bootstrap starting.
I20260811 11:55:27.007200 8445 tablet_bootstrap.cc:492] T bf3091efeb30476087adf42e8f4299bb P 19e6de45be384403b1cfa03055276e92: Bootstrap starting.
I20260811 11:55:27.007544 8446 tablet_bootstrap.cc:492] T bf3091efeb30476087adf42e8f4299bb P a141f65cdfe7445d80052c6d6976fa72: Bootstrap starting.
I20260811 11:55:27.008026 8444 tablet_bootstrap.cc:654] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:27.008073 8445 tablet_bootstrap.cc:654] T bf3091efeb30476087adf42e8f4299bb P 19e6de45be384403b1cfa03055276e92: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:27.008576 8446 tablet_bootstrap.cc:654] T bf3091efeb30476087adf42e8f4299bb P a141f65cdfe7445d80052c6d6976fa72: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:27.009279 8444 tablet_bootstrap.cc:492] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1: No bootstrap required, opened a new log
I20260811 11:55:27.009369 8444 ts_tablet_manager.cc:1397] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260811 11:55:27.009416 8445 tablet_bootstrap.cc:492] T bf3091efeb30476087adf42e8f4299bb P 19e6de45be384403b1cfa03055276e92: No bootstrap required, opened a new log
I20260811 11:55:27.009491 8445 ts_tablet_manager.cc:1397] T bf3091efeb30476087adf42e8f4299bb P 19e6de45be384403b1cfa03055276e92: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 11:55:27.009806 8446 tablet_bootstrap.cc:492] T bf3091efeb30476087adf42e8f4299bb P a141f65cdfe7445d80052c6d6976fa72: No bootstrap required, opened a new log
I20260811 11:55:27.009806 8444 raft_consensus.cc:348] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } }
I20260811 11:55:27.009845 8445 raft_consensus.cc:348] T bf3091efeb30476087adf42e8f4299bb P 19e6de45be384403b1cfa03055276e92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } }
I20260811 11:55:27.009927 8446 ts_tablet_manager.cc:1397] T bf3091efeb30476087adf42e8f4299bb P a141f65cdfe7445d80052c6d6976fa72: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:27.009934 8444 raft_consensus.cc:374] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:27.009961 8445 raft_consensus.cc:374] T bf3091efeb30476087adf42e8f4299bb P 19e6de45be384403b1cfa03055276e92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:27.009972 8444 raft_consensus.cc:729] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45df06e29b6248538b2691a04408c6b1, State: Initialized, Role: FOLLOWER
I20260811 11:55:27.010003 8445 raft_consensus.cc:729] T bf3091efeb30476087adf42e8f4299bb P 19e6de45be384403b1cfa03055276e92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19e6de45be384403b1cfa03055276e92, State: Initialized, Role: FOLLOWER
I20260811 11:55:27.010105 8444 consensus_queue.cc:260] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } }
I20260811 11:55:27.010128 8445 consensus_queue.cc:260] T bf3091efeb30476087adf42e8f4299bb P 19e6de45be384403b1cfa03055276e92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } }
I20260811 11:55:27.010397 8445 ts_tablet_manager.cc:1428] T bf3091efeb30476087adf42e8f4299bb P 19e6de45be384403b1cfa03055276e92: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 11:55:27.010394 8446 raft_consensus.cc:348] T bf3091efeb30476087adf42e8f4299bb P a141f65cdfe7445d80052c6d6976fa72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } }
I20260811 11:55:27.010464 8444 ts_tablet_manager.cc:1428] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 11:55:27.010511 8446 raft_consensus.cc:374] T bf3091efeb30476087adf42e8f4299bb P a141f65cdfe7445d80052c6d6976fa72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:27.010448 8450 tablet_replica.cc:439] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } } }
I20260811 11:55:27.010543 8446 raft_consensus.cc:729] T bf3091efeb30476087adf42e8f4299bb P a141f65cdfe7445d80052c6d6976fa72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a141f65cdfe7445d80052c6d6976fa72, State: Initialized, Role: FOLLOWER
I20260811 11:55:27.010601 8450 tablet_replica.cc:442] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:55:27.010666 8446 consensus_queue.cc:260] T bf3091efeb30476087adf42e8f4299bb P a141f65cdfe7445d80052c6d6976fa72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } }
I20260811 11:55:27.010962 8446 ts_tablet_manager.cc:1428] T bf3091efeb30476087adf42e8f4299bb P a141f65cdfe7445d80052c6d6976fa72: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:55:27.010907 8452 tablet_replica.cc:439] T bf3091efeb30476087adf42e8f4299bb P a141f65cdfe7445d80052c6d6976fa72: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } } }
I20260811 11:55:27.011044 8452 tablet_replica.cc:442] T bf3091efeb30476087adf42e8f4299bb P a141f65cdfe7445d80052c6d6976fa72: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:55:27.011168 8465 tablet_replica.cc:439] T bf3091efeb30476087adf42e8f4299bb P 19e6de45be384403b1cfa03055276e92: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } } }
I20260811 11:55:27.011310 8465 tablet_replica.cc:442] T bf3091efeb30476087adf42e8f4299bb P 19e6de45be384403b1cfa03055276e92: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:55:27.135694 8450 raft_consensus.cc:482] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:55:27.135865 8450 raft_consensus.cc:504] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } }
I20260811 11:55:27.136353 8450 leader_election.cc:288] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 19e6de45be384403b1cfa03055276e92 (127.6.143.129:37275), a141f65cdfe7445d80052c6d6976fa72 (127.6.143.130:36879)
I20260811 11:55:27.136781 8219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf3091efeb30476087adf42e8f4299bb" candidate_uuid: "45df06e29b6248538b2691a04408c6b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a141f65cdfe7445d80052c6d6976fa72" is_pre_election: true
I20260811 11:55:27.136806 8074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf3091efeb30476087adf42e8f4299bb" candidate_uuid: "45df06e29b6248538b2691a04408c6b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19e6de45be384403b1cfa03055276e92" is_pre_election: true
I20260811 11:55:27.136926 8219 raft_consensus.cc:2454] T bf3091efeb30476087adf42e8f4299bb P a141f65cdfe7445d80052c6d6976fa72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45df06e29b6248538b2691a04408c6b1 in term 0.
I20260811 11:55:27.137030 8074 raft_consensus.cc:2454] T bf3091efeb30476087adf42e8f4299bb P 19e6de45be384403b1cfa03055276e92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 45df06e29b6248538b2691a04408c6b1 in term 0.
I20260811 11:55:27.137270 8298 leader_election.cc:302] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45df06e29b6248538b2691a04408c6b1, a141f65cdfe7445d80052c6d6976fa72; no voters:
I20260811 11:55:27.137467 8450 raft_consensus.cc:2781] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:55:27.137529 8450 raft_consensus.cc:482] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:55:27.137557 8450 raft_consensus.cc:3037] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:27.138513 8450 raft_consensus.cc:504] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } }
I20260811 11:55:27.138899 8450 leader_election.cc:288] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1 [CANDIDATE]: Term 1 election: Requested vote from peers 19e6de45be384403b1cfa03055276e92 (127.6.143.129:37275), a141f65cdfe7445d80052c6d6976fa72 (127.6.143.130:36879)
I20260811 11:55:27.139245 8074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf3091efeb30476087adf42e8f4299bb" candidate_uuid: "45df06e29b6248538b2691a04408c6b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19e6de45be384403b1cfa03055276e92"
I20260811 11:55:27.139267 8219 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf3091efeb30476087adf42e8f4299bb" candidate_uuid: "45df06e29b6248538b2691a04408c6b1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a141f65cdfe7445d80052c6d6976fa72"
I20260811 11:55:27.139374 8074 raft_consensus.cc:3037] T bf3091efeb30476087adf42e8f4299bb P 19e6de45be384403b1cfa03055276e92 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:27.139374 8219 raft_consensus.cc:3037] T bf3091efeb30476087adf42e8f4299bb P a141f65cdfe7445d80052c6d6976fa72 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:27.140573 8074 raft_consensus.cc:2454] T bf3091efeb30476087adf42e8f4299bb P 19e6de45be384403b1cfa03055276e92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45df06e29b6248538b2691a04408c6b1 in term 1.
I20260811 11:55:27.140573 8219 raft_consensus.cc:2454] T bf3091efeb30476087adf42e8f4299bb P a141f65cdfe7445d80052c6d6976fa72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 45df06e29b6248538b2691a04408c6b1 in term 1.
I20260811 11:55:27.140847 8296 leader_election.cc:302] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 19e6de45be384403b1cfa03055276e92, 45df06e29b6248538b2691a04408c6b1; no voters:
I20260811 11:55:27.141040 8450 raft_consensus.cc:2781] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:27.141141 8450 raft_consensus.cc:686] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1 [term 1 LEADER]: Becoming Leader. State: Replica: 45df06e29b6248538b2691a04408c6b1, State: Running, Role: LEADER
I20260811 11:55:27.141279 8450 consensus_queue.cc:237] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } }
I20260811 11:55:27.141927 8477 tablet_replica.cc:439] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1: TxnStatusTablet state changed. Reason: New leader 45df06e29b6248538b2691a04408c6b1. Latest consensus state: current_term: 1 leader_uuid: "45df06e29b6248538b2691a04408c6b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } } }
I20260811 11:55:27.142057 8477 tablet_replica.cc:442] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1: This TxnStatusTablet replica's current role is: LEADER
I20260811 11:55:27.142300 8493 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 11:55:27.142499 8493 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 11:55:27.143069 7910 catalog_manager.cc:5495] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 45df06e29b6248538b2691a04408c6b1 (127.6.143.131). New cstate: current_term: 1 leader_uuid: "45df06e29b6248538b2691a04408c6b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } health_report { overall_health: HEALTHY } } }
I20260811 11:55:27.168651 8497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:27.164591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51474 (local address 127.6.143.131:32903)
0811 11:55:27.164858 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:27.164864 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:27.164878 (+ 14us) server_negotiation.cc:407] Connection header received
0811 11:55:27.164950 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:27.164952 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:27.165002 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:27.165096 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:55:27.165361 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:27.165964 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:27.167138 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:27.167375 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:27.167454 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:27.167776 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:55:27.168262 (+ 486us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:55:27.168379 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:27.168480 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 11:55:27.168527 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":101,"threads_started":1}
I20260811 11:55:27.168704 8486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:27.164461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:32903 (local address 127.0.0.1:51474)
0811 11:55:27.164694 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:27.164711 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:27.164819 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:27.165133 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:27.165137 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:27.165146 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 11:55:27.165255 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:27.165261 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:27.166073 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:27.166076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:27.167022 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:27.167034 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:27.167481 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:27.167485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:27.167585 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:27.167642 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 11:55:27.168372 (+ 730us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 11:55:27.168389 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 11:55:27.168471 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 11:55:27.168563 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 11:55:27.171353 8219 raft_consensus.cc:1261] T bf3091efeb30476087adf42e8f4299bb P a141f65cdfe7445d80052c6d6976fa72 [term 1 FOLLOWER]: Refusing update from remote peer 45df06e29b6248538b2691a04408c6b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:27.171578 8074 raft_consensus.cc:1261] T bf3091efeb30476087adf42e8f4299bb P 19e6de45be384403b1cfa03055276e92 [term 1 FOLLOWER]: Refusing update from remote peer 45df06e29b6248538b2691a04408c6b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:27.171847 8477 consensus_queue.cc:1035] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:27.172097 8450 consensus_queue.cc:1035] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:27.173625 8452 tablet_replica.cc:439] T bf3091efeb30476087adf42e8f4299bb P a141f65cdfe7445d80052c6d6976fa72: TxnStatusTablet state changed. Reason: New leader 45df06e29b6248538b2691a04408c6b1. Latest consensus state: current_term: 1 leader_uuid: "45df06e29b6248538b2691a04408c6b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } } }
I20260811 11:55:27.173753 8452 tablet_replica.cc:442] T bf3091efeb30476087adf42e8f4299bb P a141f65cdfe7445d80052c6d6976fa72: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:55:27.174340 8465 tablet_replica.cc:439] T bf3091efeb30476087adf42e8f4299bb P 19e6de45be384403b1cfa03055276e92: TxnStatusTablet state changed. Reason: New leader 45df06e29b6248538b2691a04408c6b1. Latest consensus state: current_term: 1 leader_uuid: "45df06e29b6248538b2691a04408c6b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } } }
I20260811 11:55:27.174472 8465 tablet_replica.cc:442] T bf3091efeb30476087adf42e8f4299bb P 19e6de45be384403b1cfa03055276e92: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:55:27.177471 8450 tablet_replica.cc:439] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "45df06e29b6248538b2691a04408c6b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } } }
I20260811 11:55:27.177575 8450 tablet_replica.cc:442] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1: This TxnStatusTablet replica's current role is: LEADER
I20260811 11:55:27.179040 8450 tablet_replica.cc:439] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "45df06e29b6248538b2691a04408c6b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } } }
I20260811 11:55:27.179145 8450 tablet_replica.cc:442] T bf3091efeb30476087adf42e8f4299bb P 45df06e29b6248538b2691a04408c6b1: This TxnStatusTablet replica's current role is: LEADER
I20260811 11:55:27.179303 8452 tablet_replica.cc:439] T bf3091efeb30476087adf42e8f4299bb P a141f65cdfe7445d80052c6d6976fa72: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "45df06e29b6248538b2691a04408c6b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } } }
I20260811 11:55:27.179409 8452 tablet_replica.cc:442] T bf3091efeb30476087adf42e8f4299bb P a141f65cdfe7445d80052c6d6976fa72: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:55:27.179601 8499 mvcc.cc:205] Tried to move back new op lower bound from 7317296444088954880 to 7317296443971981312. Current Snapshot: MvccSnapshot[applied={T|T < 7317296444088954880}]
I20260811 11:55:27.181183 8465 tablet_replica.cc:439] T bf3091efeb30476087adf42e8f4299bb P 19e6de45be384403b1cfa03055276e92: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "45df06e29b6248538b2691a04408c6b1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 } } peers { permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 } } peers { permanent_uuid: "45df06e29b6248538b2691a04408c6b1" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 32903 } } }
I20260811 11:55:27.181293 8465 tablet_replica.cc:442] T bf3091efeb30476087adf42e8f4299bb P 19e6de45be384403b1cfa03055276e92: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:55:27.186939 8497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:27.183614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51480 (local address 127.6.143.131:32903)
0811 11:55:27.183725 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:27.183729 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:27.183870 (+ 141us) server_negotiation.cc:407] Connection header received
0811 11:55:27.184039 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:27.184041 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:27.184078 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:27.184139 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:55:27.184403 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:27.184843 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:27.186000 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:27.186201 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:27.186289 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:27.186555 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:55:27.186615 (+ 60us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:55:27.186663 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:27.186779 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 11:55:27.186817 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 11:55:27.197332 8497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:27.194114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:44893 (local address 127.6.143.131:32903)
0811 11:55:27.194224 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:27.194230 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:27.194391 (+ 161us) server_negotiation.cc:407] Connection header received
0811 11:55:27.194514 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:27.194519 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:27.194560 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:27.194638 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:55:27.194877 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:27.195338 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:27.196279 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:27.196485 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:27.196593 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:27.196859 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:55:27.196918 (+ 59us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:55:27.196999 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:27.197082 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 11:55:27.197180 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":37}
I20260811 11:55:27.197356 8510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:27.194037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:32903 (local address 127.6.143.131:44893)
0811 11:55:27.194316 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:27.194326 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:27.194438 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:27.194645 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:27.194647 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:27.194654 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 11:55:27.194781 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:27.194785 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:27.195449 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:27.195455 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:27.196163 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:27.196173 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:27.196591 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:27.196594 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:27.196692 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:27.196726 (+ 34us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 11:55:27.197002 (+ 276us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 11:55:27.197010 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 11:55:27.197074 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 11:55:27.197202 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":34,"thread_start_us":86,"threads_started":1}
I20260811 11:55:27.203385 8219 raft_consensus.cc:1261] T 3a4ab234b3544900a55086271f9ce2bd P a141f65cdfe7445d80052c6d6976fa72 [term 1 FOLLOWER]: Refusing update from remote peer 45df06e29b6248538b2691a04408c6b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:27.203474 8074 raft_consensus.cc:1261] T 3a4ab234b3544900a55086271f9ce2bd P 19e6de45be384403b1cfa03055276e92 [term 1 FOLLOWER]: Refusing update from remote peer 45df06e29b6248538b2691a04408c6b1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:27.203806 8477 consensus_queue.cc:1035] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "19e6de45be384403b1cfa03055276e92" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 37275 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:27.204049 8450 consensus_queue.cc:1035] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a141f65cdfe7445d80052c6d6976fa72" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:27.245447 7910 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:55:27.245610 7910 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33982:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:55:27.247567 7910 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 64e8c570ad8a4963935c53b05a7e3bd6: Sending DeleteTablet for 3 replicas of tablet 3a4ab234b3544900a55086271f9ce2bd
I20260811 11:55:27.248117 8053 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a4ab234b3544900a55086271f9ce2bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:27 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:40880
I20260811 11:55:27.248198 8348 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a4ab234b3544900a55086271f9ce2bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:27 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:51466
I20260811 11:55:27.248193 8200 tablet_service.cc:1514] Processing DeleteTablet for tablet 3a4ab234b3544900a55086271f9ce2bd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:27 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:54448
I20260811 11:55:27.248528 8518 tablet_replica.cc:330] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1: stopping tablet replica
I20260811 11:55:27.248569 8517 tablet_replica.cc:330] T 3a4ab234b3544900a55086271f9ce2bd P 19e6de45be384403b1cfa03055276e92: stopping tablet replica
I20260811 11:55:27.248649 8518 raft_consensus.cc:2229] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:55:27.248682 8517 raft_consensus.cc:2229] T 3a4ab234b3544900a55086271f9ce2bd P 19e6de45be384403b1cfa03055276e92 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:55:27.248787 8517 raft_consensus.cc:2258] T 3a4ab234b3544900a55086271f9ce2bd P 19e6de45be384403b1cfa03055276e92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:27.248914 8518 raft_consensus.cc:2258] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:27.249477 8517 ts_tablet_manager.cc:1905] T 3a4ab234b3544900a55086271f9ce2bd P 19e6de45be384403b1cfa03055276e92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:27.249673 8518 ts_tablet_manager.cc:1905] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:27.252003 8517 ts_tablet_manager.cc:1918] T 3a4ab234b3544900a55086271f9ce2bd P 19e6de45be384403b1cfa03055276e92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 11:55:27.252095 8517 log.cc:1196] T 3a4ab234b3544900a55086271f9ce2bd P 19e6de45be384403b1cfa03055276e92: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-0/wal/wals/3a4ab234b3544900a55086271f9ce2bd
I20260811 11:55:27.252373 8517 ts_tablet_manager.cc:1939] T 3a4ab234b3544900a55086271f9ce2bd P 19e6de45be384403b1cfa03055276e92: Deleting consensus metadata
I20260811 11:55:27.252532 8518 ts_tablet_manager.cc:1918] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 11:55:27.252610 8518 log.cc:1196] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-2/wal/wals/3a4ab234b3544900a55086271f9ce2bd
I20260811 11:55:27.252897 8518 ts_tablet_manager.cc:1939] T 3a4ab234b3544900a55086271f9ce2bd P 45df06e29b6248538b2691a04408c6b1: Deleting consensus metadata
I20260811 11:55:27.253029 7897 catalog_manager.cc:4841] TS 19e6de45be384403b1cfa03055276e92 (127.6.143.129:37275): tablet 3a4ab234b3544900a55086271f9ce2bd (table test-table [id=6a5571282d2248ab8e6e73cc166ba9d1]) successfully deleted
I20260811 11:55:27.253557 8519 tablet_replica.cc:330] T 3a4ab234b3544900a55086271f9ce2bd P a141f65cdfe7445d80052c6d6976fa72: stopping tablet replica
I20260811 11:55:27.253572 7897 catalog_manager.cc:4841] TS 45df06e29b6248538b2691a04408c6b1 (127.6.143.131:32903): tablet 3a4ab234b3544900a55086271f9ce2bd (table test-table [id=6a5571282d2248ab8e6e73cc166ba9d1]) successfully deleted
I20260811 11:55:27.253705 8519 raft_consensus.cc:2229] T 3a4ab234b3544900a55086271f9ce2bd P a141f65cdfe7445d80052c6d6976fa72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:55:27.253818 8519 raft_consensus.cc:2258] T 3a4ab234b3544900a55086271f9ce2bd P a141f65cdfe7445d80052c6d6976fa72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:27.254444 8519 ts_tablet_manager.cc:1905] T 3a4ab234b3544900a55086271f9ce2bd P a141f65cdfe7445d80052c6d6976fa72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:27.257180 8519 ts_tablet_manager.cc:1918] T 3a4ab234b3544900a55086271f9ce2bd P a141f65cdfe7445d80052c6d6976fa72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 11:55:27.257263 8519 log.cc:1196] T 3a4ab234b3544900a55086271f9ce2bd P a141f65cdfe7445d80052c6d6976fa72: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786449319379472-6718-0/minicluster-data/ts-1/wal/wals/3a4ab234b3544900a55086271f9ce2bd
I20260811 11:55:27.257519 8519 ts_tablet_manager.cc:1939] T 3a4ab234b3544900a55086271f9ce2bd P a141f65cdfe7445d80052c6d6976fa72: Deleting consensus metadata
I20260811 11:55:27.258175 7899 catalog_manager.cc:4841] TS a141f65cdfe7445d80052c6d6976fa72 (127.6.143.130:36879): tablet 3a4ab234b3544900a55086271f9ce2bd (table test-table [id=6a5571282d2248ab8e6e73cc166ba9d1]) successfully deleted
Aug 11 11:55:27 dist-test-slave-zkpd krb5kdc[7865](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449326, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.143.129@KRBTEST.COM
I20260811 11:55:27.263734 8443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:27.252419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40902 (local address 127.6.143.129:37275)
0811 11:55:27.252564 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:27.252569 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:27.253305 (+ 736us) server_negotiation.cc:407] Connection header received
0811 11:55:27.253538 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:27.253541 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:27.253593 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:27.253651 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:27.254636 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:27.255142 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:27.256034 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:27.256249 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:27.256360 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:27.260065 (+ 3705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:27.260075 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:27.260087 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:27.260109 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:27.261955 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:27.262457 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:27.262462 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:27.262463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:27.262545 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:27.262829 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:27.262831 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:27.262833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:27.263067 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:27.263217 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:27.263482 (+ 265us) server_negotiation.cc:299] Negotiation successful
0811 11:55:27.263581 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 11:55:27.264281 8115 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40902
I20260811 11:55:27.274962 8487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:27.265961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34010 (local address 127.6.143.190:39261)
0811 11:55:27.266082 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:27.266086 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:27.266173 (+ 87us) server_negotiation.cc:407] Connection header received
0811 11:55:27.266266 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:27.266270 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:27.266326 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:27.266415 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:27.267123 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:27.267980 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:27.268799 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:27.269178 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:27.269293 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:27.270741 (+ 1448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:27.270746 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:27.270750 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:27.270777 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:27.272641 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:27.273493 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:27.273497 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:27.273499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:27.273594 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:27.273946 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:27.273950 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:27.273952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:27.274176 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:27.274344 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:27.274614 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 11:55:27.274809 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 11:55:27.275334 7910 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34010
I20260811 11:55:27.276767 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 7987
I20260811 11:55:27.290482 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 8130
I20260811 11:55:27.301426 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 8279
I20260811 11:55:27.310289 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 7878
2026-08-11T11:55:27Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3176 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:55:27 dist-test-slave-zkpd krb5kdc[8526](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:55:27 dist-test-slave-zkpd krb5kdc[8526](info): set up 2 sockets
Aug 11 11:55:27 dist-test-slave-zkpd krb5kdc[8526](info): commencing operation
krb5kdc: starting...
W20260811 11:55:28.390095 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:55:28 dist-test-slave-zkpd krb5kdc[8526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:55:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:55:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
I20260811 11:55:28.514575 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:45171
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:32809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:45171
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:28.626462 8539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:28.626716 8539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:28.626755 8539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:28.630723 8539 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:55:28.630780 8539 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 11:55:28.630797 8539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:28.630816 8539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:28.630831 8539 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:55:28.630846 8539 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:55:28.636059 8539 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:32809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:45171
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:45171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:28.637056 8539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:28.638012 8539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:28.646462 8547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:28.646481 8544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:28.646658 8539 server_base.cc:1034] running on GCE node
W20260811 11:55:28.646474 8545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:28.647342 8539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:28.648466 8539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:28.649629 8539 hybrid_clock.cc:648] HybridClock initialized: now 1786449328649596 us; error 44 us; skew 500 ppm
Aug 11 11:55:28 dist-test-slave-zkpd krb5kdc[8526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449328, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:28.652418 8539 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:55:28.653323 8539 webserver.cc:460] Webserver started at http://127.6.143.190:39653/ using document root <none> and password file <none>
I20260811 11:55:28.653867 8539 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:28.653927 8539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:28.654088 8539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:28.656152 8539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "016ab2aae5fd45419161a6186a9719b6"
format_stamp: "Formatted at 2026-08-11 11:55:28 on dist-test-slave-zkpd"
server_key: "31e5c549e8754de3ce2ae6b7a88afa2f"
server_key_iv: "615be4e58489c261b064d64e131d08b9"
server_key_version: "encryptionkey@0"
I20260811 11:55:28.656661 8539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "016ab2aae5fd45419161a6186a9719b6"
format_stamp: "Formatted at 2026-08-11 11:55:28 on dist-test-slave-zkpd"
server_key: "31e5c549e8754de3ce2ae6b7a88afa2f"
server_key_iv: "615be4e58489c261b064d64e131d08b9"
server_key_version: "encryptionkey@0"
I20260811 11:55:28.660946 8539 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 11:55:28.663223 8554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:28.664527 8539 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:28.664673 8539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "016ab2aae5fd45419161a6186a9719b6"
format_stamp: "Formatted at 2026-08-11 11:55:28 on dist-test-slave-zkpd"
server_key: "31e5c549e8754de3ce2ae6b7a88afa2f"
server_key_iv: "615be4e58489c261b064d64e131d08b9"
server_key_version: "encryptionkey@0"
I20260811 11:55:28.664772 8539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:28.681540 8539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:28.684396 8539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:28.684588 8539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:28.692631 8539 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:45171
I20260811 11:55:28.692657 8620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:45171 every 8 connection(s)
I20260811 11:55:28.693526 8539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:55:28.696233 8622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:28.701047 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 8539
I20260811 11:55:28.701145 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:55:28.701426 6718 external_mini_cluster.cc:1428] Setting key 1bcfef63c25f67c9e400cc9d82a0d005
I20260811 11:55:28.701756 8622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 016ab2aae5fd45419161a6186a9719b6: Bootstrap starting.
I20260811 11:55:28.704386 8622 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 016ab2aae5fd45419161a6186a9719b6: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:28.705286 8622 log.cc:824] T 00000000000000000000000000000000 P 016ab2aae5fd45419161a6186a9719b6: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:28.707414 8622 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 016ab2aae5fd45419161a6186a9719b6: No bootstrap required, opened a new log
Aug 11 11:55:28 dist-test-slave-zkpd krb5kdc[8526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:28.711216 8622 raft_consensus.cc:348] T 00000000000000000000000000000000 P 016ab2aae5fd45419161a6186a9719b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "016ab2aae5fd45419161a6186a9719b6" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 45171 } }
I20260811 11:55:28.711534 8622 raft_consensus.cc:374] T 00000000000000000000000000000000 P 016ab2aae5fd45419161a6186a9719b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:28.711629 8622 raft_consensus.cc:729] T 00000000000000000000000000000000 P 016ab2aae5fd45419161a6186a9719b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016ab2aae5fd45419161a6186a9719b6, State: Initialized, Role: FOLLOWER
I20260811 11:55:28.712129 8622 consensus_queue.cc:260] T 00000000000000000000000000000000 P 016ab2aae5fd45419161a6186a9719b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "016ab2aae5fd45419161a6186a9719b6" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 45171 } }
I20260811 11:55:28.712282 8622 raft_consensus.cc:388] T 00000000000000000000000000000000 P 016ab2aae5fd45419161a6186a9719b6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:55:28.712387 8622 raft_consensus.cc:482] T 00000000000000000000000000000000 P 016ab2aae5fd45419161a6186a9719b6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:55:28.712520 8622 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 016ab2aae5fd45419161a6186a9719b6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:28.713338 8622 raft_consensus.cc:504] T 00000000000000000000000000000000 P 016ab2aae5fd45419161a6186a9719b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "016ab2aae5fd45419161a6186a9719b6" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 45171 } }
I20260811 11:55:28.713694 8622 leader_election.cc:302] T 00000000000000000000000000000000 P 016ab2aae5fd45419161a6186a9719b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 016ab2aae5fd45419161a6186a9719b6; no voters:
I20260811 11:55:28.714021 8622 leader_election.cc:288] T 00000000000000000000000000000000 P 016ab2aae5fd45419161a6186a9719b6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:55:28.714089 8628 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 016ab2aae5fd45419161a6186a9719b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:28.714288 8628 raft_consensus.cc:686] T 00000000000000000000000000000000 P 016ab2aae5fd45419161a6186a9719b6 [term 1 LEADER]: Becoming Leader. State: Replica: 016ab2aae5fd45419161a6186a9719b6, State: Running, Role: LEADER
I20260811 11:55:28.714632 8628 consensus_queue.cc:237] T 00000000000000000000000000000000 P 016ab2aae5fd45419161a6186a9719b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "016ab2aae5fd45419161a6186a9719b6" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 45171 } }
I20260811 11:55:28.715278 8622 sys_catalog.cc:564] T 00000000000000000000000000000000 P 016ab2aae5fd45419161a6186a9719b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:55:28.716385 8625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:28.702826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50482 (local address 127.6.143.190:45171)
0811 11:55:28.703262 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:28.703280 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:28.703323 (+ 43us) server_negotiation.cc:407] Connection header received
0811 11:55:28.704158 (+ 835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:28.704182 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:28.704610 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:28.704982 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:28.705688 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:28.707049 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:28.708028 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:28.708685 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:28.708879 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:28.711191 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:28.711198 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:28.711215 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:28.711248 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:28.713214 (+ 1966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:28.713741 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:28.713747 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:28.713753 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:28.713949 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:28.714236 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:28.714239 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:28.714241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:28.714723 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:28.714927 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:28.715193 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 11:55:28.715493 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":117,"threads_started":1}
I20260811 11:55:28.716512 8629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 016ab2aae5fd45419161a6186a9719b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "016ab2aae5fd45419161a6186a9719b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "016ab2aae5fd45419161a6186a9719b6" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 45171 } } }
I20260811 11:55:28.716691 8629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 016ab2aae5fd45419161a6186a9719b6 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:28.716746 8630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 016ab2aae5fd45419161a6186a9719b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 016ab2aae5fd45419161a6186a9719b6. Latest consensus state: current_term: 1 leader_uuid: "016ab2aae5fd45419161a6186a9719b6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "016ab2aae5fd45419161a6186a9719b6" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 45171 } } }
I20260811 11:55:28.716817 8630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 016ab2aae5fd45419161a6186a9719b6 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:28.717151 8638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:55:28.720254 8638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:55:28.725021 8638 catalog_manager.cc:1347] Generated new cluster ID: e542f87fbbd64e72be22fcf747dd2f94
I20260811 11:55:28.725085 8638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:55:28.753916 8638 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:55:28.754850 8638 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:55:28.765434 8638 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 016ab2aae5fd45419161a6186a9719b6: Generated new TSK 0
I20260811 11:55:28.766122 8638 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
I20260811 11:55:28.803215 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:0
--local_ip_for_outbound_sockets=127.6.143.129
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:45171
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:32809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:28.915921 8652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:28.916194 8652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:28.916270 8652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:28.920375 8652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:28.920425 8652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:28.920496 8652 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 11:55:28.920532 8652 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 11:55:28.920603 8652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:55:28.925603 8652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:32809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.143.190:45171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:28.926549 8652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:28.927526 8652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:28.936971 8660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:28.937006 8657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:28.937026 8652 server_base.cc:1034] running on GCE node
W20260811 11:55:28.937256 8658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:28.937659 8652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:28.938274 8652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:28.939469 8652 hybrid_clock.cc:648] HybridClock initialized: now 1786449328939447 us; error 32 us; skew 500 ppm
Aug 11 11:55:28 dist-test-slave-zkpd krb5kdc[8526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449328, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:28.942200 8652 init.cc:377] Logged in from keytab as kudu/127.6.143.129@KRBTEST.COM (short username kudu)
I20260811 11:55:28.943017 8652 webserver.cc:460] Webserver started at http://127.6.143.129:45593/ using document root <none> and password file <none>
I20260811 11:55:28.943572 8652 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:28.943665 8652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:28.943940 8652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:28.945895 8652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-0/data/instance:
uuid: "166d007d7628419abf3b0f98ac59a04f"
format_stamp: "Formatted at 2026-08-11 11:55:28 on dist-test-slave-zkpd"
server_key: "dec31a44d5d1063a0fc6e945c67ddf36"
server_key_iv: "27d08ffe14c6ba120b65181947f3cb3f"
server_key_version: "encryptionkey@0"
I20260811 11:55:28.946480 8652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance:
uuid: "166d007d7628419abf3b0f98ac59a04f"
format_stamp: "Formatted at 2026-08-11 11:55:28 on dist-test-slave-zkpd"
server_key: "dec31a44d5d1063a0fc6e945c67ddf36"
server_key_iv: "27d08ffe14c6ba120b65181947f3cb3f"
server_key_version: "encryptionkey@0"
I20260811 11:55:28.950133 8652 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260811 11:55:28.952512 8668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:28.953842 8652 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:55:28.953992 8652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "166d007d7628419abf3b0f98ac59a04f"
format_stamp: "Formatted at 2026-08-11 11:55:28 on dist-test-slave-zkpd"
server_key: "dec31a44d5d1063a0fc6e945c67ddf36"
server_key_iv: "27d08ffe14c6ba120b65181947f3cb3f"
server_key_version: "encryptionkey@0"
I20260811 11:55:28.954104 8652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:28.964789 8652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:28.967409 8652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:28.967604 8652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:28.969233 8652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:28.969333 8652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:28.969419 8652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:28.969487 8652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 11:55:28 dist-test-slave-zkpd krb5kdc[8526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449328, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:28.980685 8652 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:45063
I20260811 11:55:28.980741 8789 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:45063 every 8 connection(s)
I20260811 11:55:28.981750 8652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:55:28.983065 8625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:28.969975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:56791 (local address 127.6.143.190:45171)
0811 11:55:28.970138 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:28.970145 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:28.971444 (+ 1299us) server_negotiation.cc:407] Connection header received
0811 11:55:28.972569 (+ 1125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:28.972572 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:28.972631 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:28.972713 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:28.973786 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:28.974316 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:28.975579 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:28.975945 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:28.976071 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:28.978822 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:28.978830 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:28.978832 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:28.978857 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:28.980623 (+ 1766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:28.981153 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:28.981157 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:28.981159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:28.981266 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:28.981608 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:28.981611 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:28.981613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:28.981853 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:28.981959 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:28.982784 (+ 825us) server_negotiation.cc:299] Negotiation successful
0811 11:55:28.982911 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 11:55:28.983630 8677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:28.970816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:45171 (local address 127.6.143.129:56791)
0811 11:55:28.971266 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:28.971315 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:28.972318 (+ 1003us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:28.972875 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:28.972882 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:28.973188 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:28.973622 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:28.973634 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:28.974464 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:28.974467 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:28.975437 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:28.975448 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:28.976116 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:28.976121 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:28.976285 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:28.976877 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:28.976901 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:28.978653 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:28.980733 (+ 2080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:28.980740 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:28.980755 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:28.981048 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:28.981370 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:28.981372 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:28.981373 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:28.981509 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:28.981962 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:28.981967 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:28.982235 (+ 268us) client_negotiation.cc:769] Sending connection context
0811 11:55:28.982444 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 11:55:28.982752 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":132,"threads_started":1}
I20260811 11:55:28.984908 8790 heartbeater.cc:347] Connected to a master server at 127.6.143.190:45171
I20260811 11:55:28.985045 8790 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:28.985565 8790 heartbeater.cc:511] Master 127.6.143.190:45171 requested a full tablet report, sending...
I20260811 11:55:28.987195 8571 ts_manager.cc:194] Registered new tserver with Master: 166d007d7628419abf3b0f98ac59a04f (127.6.143.129:45063)
I20260811 11:55:28.988399 8571 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.129@KRBTEST.COM'} at 127.6.143.129:56791
I20260811 11:55:28.989548 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 8652
I20260811 11:55:28.989681 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance
I20260811 11:55:28.990046 6718 external_mini_cluster.cc:1428] Setting key f4e9306efffb2c1025ecc36fec57f51c
I20260811 11:55:28.994902 8625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:28.987059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:33245 (local address 127.6.143.190:45171)
0811 11:55:28.987175 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:28.987179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:28.987354 (+ 175us) server_negotiation.cc:407] Connection header received
0811 11:55:28.987527 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:28.987534 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:28.987573 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:28.987654 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:28.988145 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:28.988680 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:28.989859 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:28.990178 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:28.990292 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:28.991626 (+ 1334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:28.991630 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:28.991633 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:28.991658 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:28.993298 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:28.993720 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:28.993726 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:28.993731 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:28.993822 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:28.994163 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:28.994166 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:28.994167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:28.994324 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:28.994407 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:28.994703 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 11:55:28.994794 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 11:55:28.995035 8796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:28.986990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:45171 (local address 127.6.143.129:33245)
0811 11:55:28.987260 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:28.987272 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:28.987422 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:28.987670 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:28.987673 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:28.987875 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:28.988034 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:28.988040 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:28.988804 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:28.988807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:28.989677 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:28.989693 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:28.990312 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:28.990315 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:28.990607 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:28.991080 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:28.991093 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:28.991485 (+ 392us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:28.993421 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:28.993424 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:28.993426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:28.993614 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:28.993934 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:28.993937 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:28.993939 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:28.994055 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:28.994420 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:28.994425 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:28.994534 (+ 109us) client_negotiation.cc:769] Sending connection context
0811 11:55:28.994652 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 11:55:28.994809 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":110,"threads_started":1}
WARNING: no policy specified for kudu/127.6.143.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
I20260811 11:55:29.016841 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:0
--local_ip_for_outbound_sockets=127.6.143.130
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:45171
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:32809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:29.135192 8800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:29.135532 8800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:29.135632 8800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:29.140153 8800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:29.140231 8800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:29.140298 8800 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 11:55:29.140329 8800 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 11:55:29.140394 8800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:55:29.148232 8800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:32809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.143.190:45171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:29.149611 8800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:29.150956 8800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:29.163738 8806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:29.163738 8805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:29.164345 8808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:29.164336 8800 server_base.cc:1034] running on GCE node
I20260811 11:55:29.164922 8800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:29.165796 8800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:29.167001 8800 hybrid_clock.cc:648] HybridClock initialized: now 1786449329166979 us; error 32 us; skew 500 ppm
Aug 11 11:55:29 dist-test-slave-zkpd krb5kdc[8526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449329, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:29.170804 8800 init.cc:377] Logged in from keytab as kudu/127.6.143.130@KRBTEST.COM (short username kudu)
I20260811 11:55:29.171655 8800 webserver.cc:460] Webserver started at http://127.6.143.130:43723/ using document root <none> and password file <none>
I20260811 11:55:29.172420 8800 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:29.172503 8800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:29.172741 8800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:29.174847 8800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-1/data/instance:
uuid: "fed2e14718264870afb1a78b04a64ee7"
format_stamp: "Formatted at 2026-08-11 11:55:29 on dist-test-slave-zkpd"
server_key: "502c1344329072dbcec9922813e25131"
server_key_iv: "3de9aad7c45c671d5845299ea2879d93"
server_key_version: "encryptionkey@0"
I20260811 11:55:29.175432 8800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance:
uuid: "fed2e14718264870afb1a78b04a64ee7"
format_stamp: "Formatted at 2026-08-11 11:55:29 on dist-test-slave-zkpd"
server_key: "502c1344329072dbcec9922813e25131"
server_key_iv: "3de9aad7c45c671d5845299ea2879d93"
server_key_version: "encryptionkey@0"
I20260811 11:55:29.179224 8800 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 11:55:29.181602 8816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:29.182782 8800 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:29.182936 8800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "fed2e14718264870afb1a78b04a64ee7"
format_stamp: "Formatted at 2026-08-11 11:55:29 on dist-test-slave-zkpd"
server_key: "502c1344329072dbcec9922813e25131"
server_key_iv: "3de9aad7c45c671d5845299ea2879d93"
server_key_version: "encryptionkey@0"
I20260811 11:55:29.183046 8800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:29.195349 8800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:29.198474 8800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:29.198668 8800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:29.200099 8800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:29.200166 8800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:29.200230 8800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:29.200280 8800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 11:55:29 dist-test-slave-zkpd krb5kdc[8526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449329, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:29.211517 8800 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:36051
I20260811 11:55:29.211639 8940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:36051 every 8 connection(s)
I20260811 11:55:29.212672 8800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
I20260811 11:55:29.213284 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 8800
I20260811 11:55:29.213436 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance
I20260811 11:55:29.213783 6718 external_mini_cluster.cc:1428] Setting key 7a06396e18ba58f1e4e3b80239c87b1b
I20260811 11:55:29.215165 8625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.200975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:41295 (local address 127.6.143.190:45171)
0811 11:55:29.201149 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:29.201156 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:29.201975 (+ 819us) server_negotiation.cc:407] Connection header received
0811 11:55:29.203245 (+ 1270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:29.203248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:29.203314 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:29.203395 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:29.205267 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.205796 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.207690 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.208049 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.208172 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.210998 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:29.211001 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:29.211004 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:29.211030 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:29.212787 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:29.213362 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:29.213368 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:29.213370 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:29.213480 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:29.213960 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:29.213964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:29.213965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:29.214197 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:29.214314 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:29.214863 (+ 549us) server_negotiation.cc:299] Negotiation successful
0811 11:55:29.214989 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 11:55:29.216144 8825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.201309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:45171 (local address 127.6.143.130:41295)
0811 11:55:29.201791 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:29.201847 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:29.202968 (+ 1121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:29.203853 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:29.203865 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:29.204385 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:29.205083 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.205106 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.205905 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.205909 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.207528 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.207543 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.208240 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.208248 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.208474 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.209021 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:29.209044 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:29.210825 (+ 1781us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:29.212896 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:29.212901 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:29.212914 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:29.213233 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:29.213595 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:29.213600 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:29.213603 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:29.213811 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:29.214336 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:29.214345 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:29.214631 (+ 286us) client_negotiation.cc:769] Sending connection context
0811 11:55:29.214788 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 11:55:29.215119 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":108,"threads_started":1}
I20260811 11:55:29.217387 8941 heartbeater.cc:347] Connected to a master server at 127.6.143.190:45171
I20260811 11:55:29.217527 8941 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:29.218045 8941 heartbeater.cc:511] Master 127.6.143.190:45171 requested a full tablet report, sending...
I20260811 11:55:29.219096 8571 ts_manager.cc:194] Registered new tserver with Master: fed2e14718264870afb1a78b04a64ee7 (127.6.143.130:36051)
I20260811 11:55:29.219731 8571 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.130@KRBTEST.COM'} at 127.6.143.130:41295
WARNING: no policy specified for kudu/127.6.143.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.131@KRBTEST.COM" created.
I20260811 11:55:29.228304 8625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.219956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:39905 (local address 127.6.143.190:45171)
0811 11:55:29.220271 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:29.220278 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:29.221770 (+ 1492us) server_negotiation.cc:407] Connection header received
0811 11:55:29.221890 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:29.221893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:29.221938 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:29.222025 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:29.222447 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.222939 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.223759 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.224043 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.224156 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.225203 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:29.225207 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:29.225209 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:29.225233 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:29.226502 (+ 1269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:29.227023 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:29.227027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:29.227028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:29.227147 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:29.227455 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:29.227459 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:29.227461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:29.227666 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:29.227780 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:29.228065 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 11:55:29.228171 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240}
I20260811 11:55:29.228317 8948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.219846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:45171 (local address 127.6.143.130:39905)
0811 11:55:29.221674 (+ 1828us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:29.221687 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:29.221810 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:29.222040 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:29.222043 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:29.222235 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:29.222323 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.222328 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.223073 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.223076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.223630 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.223638 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.224158 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.224161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.224255 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.224761 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:29.224772 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:29.225067 (+ 295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:29.226686 (+ 1619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:29.226689 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:29.226693 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:29.226916 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:29.227246 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:29.227250 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:29.227252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:29.227359 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:29.227769 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:29.227772 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:29.227872 (+ 100us) client_negotiation.cc:769] Sending connection context
0811 11:55:29.228009 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 11:55:29.228142 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":123,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
I20260811 11:55:29.243261 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:0
--local_ip_for_outbound_sockets=127.6.143.131
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:45171
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:32809
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:29.360416 8951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:29.360648 8951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:29.360695 8951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:29.364579 8951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:29.364629 8951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:29.364666 8951 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 11:55:29.364679 8951 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 11:55:29.364708 8951 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:55:29.369829 8951 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:32809
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.143.190:45171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:29.370738 8951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:29.371676 8951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:29.381315 8959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:29.381349 8957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:29.381350 8951 server_base.cc:1034] running on GCE node
W20260811 11:55:29.381515 8956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:29.381925 8951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:29.382547 8951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:29.383738 8951 hybrid_clock.cc:648] HybridClock initialized: now 1786449329383724 us; error 29 us; skew 500 ppm
Aug 11 11:55:29 dist-test-slave-zkpd krb5kdc[8526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449329, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:29.386606 8951 init.cc:377] Logged in from keytab as kudu/127.6.143.131@KRBTEST.COM (short username kudu)
I20260811 11:55:29.387557 8951 webserver.cc:460] Webserver started at http://127.6.143.131:40979/ using document root <none> and password file <none>
I20260811 11:55:29.388195 8951 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:29.388286 8951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:29.388520 8951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:29.390551 8951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-2/data/instance:
uuid: "a4d81298eb6e46bc95c4ab437d740735"
format_stamp: "Formatted at 2026-08-11 11:55:29 on dist-test-slave-zkpd"
server_key: "6cb852e0af1f5d274c628e8108228804"
server_key_iv: "a528df582127641f4bd5e2707fc52ab8"
server_key_version: "encryptionkey@0"
I20260811 11:55:29.391113 8951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance:
uuid: "a4d81298eb6e46bc95c4ab437d740735"
format_stamp: "Formatted at 2026-08-11 11:55:29 on dist-test-slave-zkpd"
server_key: "6cb852e0af1f5d274c628e8108228804"
server_key_iv: "a528df582127641f4bd5e2707fc52ab8"
server_key_version: "encryptionkey@0"
I20260811 11:55:29.394735 8951 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260811 11:55:29.397099 8966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:29.398316 8951 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 11:55:29.398453 8951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "a4d81298eb6e46bc95c4ab437d740735"
format_stamp: "Formatted at 2026-08-11 11:55:29 on dist-test-slave-zkpd"
server_key: "6cb852e0af1f5d274c628e8108228804"
server_key_iv: "a528df582127641f4bd5e2707fc52ab8"
server_key_version: "encryptionkey@0"
I20260811 11:55:29.398561 8951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:29.414348 8951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:29.417220 8951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:29.417412 8951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:29.418919 8951 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:29.419001 8951 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:29.419070 8951 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:29.419124 8951 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 11:55:29 dist-test-slave-zkpd krb5kdc[8526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449329, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:29.429855 8951 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:43925
I20260811 11:55:29.429952 9093 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:43925 every 8 connection(s)
I20260811 11:55:29.430718 8951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:55:29.433022 8625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.419707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:37245 (local address 127.6.143.190:45171)
0811 11:55:29.419889 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:29.419894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:29.421260 (+ 1366us) server_negotiation.cc:407] Connection header received
0811 11:55:29.422638 (+ 1378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:29.422641 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:29.422691 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:29.422767 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:29.424442 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.425134 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.426250 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.426491 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.426604 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.429208 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:29.429211 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:29.429216 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:29.429245 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:29.430845 (+ 1600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:29.431471 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:29.431475 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:29.431479 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:29.431572 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:29.431951 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:29.431955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:29.431957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:29.432141 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:29.432250 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:29.432726 (+ 476us) server_negotiation.cc:299] Negotiation successful
0811 11:55:29.432841 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 11:55:29.433737 8975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.420573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:45171 (local address 127.6.143.131:37245)
0811 11:55:29.421072 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:29.421133 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:29.422382 (+ 1249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:29.422973 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:29.422985 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:29.423509 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:29.424265 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.424291 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.425268 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.425271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.426120 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.426129 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.426644 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.426648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.426848 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.427408 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:29.427423 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:29.429041 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:29.430979 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:29.430984 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:29.430998 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:29.431318 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:29.431664 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:29.431667 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:29.431669 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:29.431823 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:29.432250 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:29.432258 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:29.432486 (+ 228us) client_negotiation.cc:769] Sending connection context
0811 11:55:29.432664 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 11:55:29.432966 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":106,"threads_started":1}
I20260811 11:55:29.435057 9094 heartbeater.cc:347] Connected to a master server at 127.6.143.190:45171
I20260811 11:55:29.435218 9094 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:29.435688 9094 heartbeater.cc:511] Master 127.6.143.190:45171 requested a full tablet report, sending...
I20260811 11:55:29.436789 8572 ts_manager.cc:194] Registered new tserver with Master: a4d81298eb6e46bc95c4ab437d740735 (127.6.143.131:43925)
I20260811 11:55:29.437621 8572 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.131@KRBTEST.COM'} at 127.6.143.131:37245
I20260811 11:55:29.440117 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 8951
I20260811 11:55:29.440246 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance
I20260811 11:55:29.440644 6718 external_mini_cluster.cc:1428] Setting key 469278ca8535770d6648a4ab2208a22e
I20260811 11:55:29.442802 6718 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:55:29.444679 9101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.437202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:45171 (local address 127.6.143.131:34825)
0811 11:55:29.437469 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:29.437483 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:29.437564 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:29.437805 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:29.437808 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:29.438042 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:29.438162 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.438169 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.439163 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.439165 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.439713 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.439720 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.440262 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.440267 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.440368 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.440910 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:29.440924 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:29.441246 (+ 322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:29.443072 (+ 1826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:29.443075 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:29.443077 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:29.443325 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:29.443593 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:29.443599 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:29.443602 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:29.443707 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:29.444158 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:29.444161 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:29.444260 (+ 99us) client_negotiation.cc:769] Sending connection context
0811 11:55:29.444368 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 11:55:29.444505 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":93,"threads_started":1}
I20260811 11:55:29.444679 8625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.437475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:34825 (local address 127.6.143.190:45171)
0811 11:55:29.437571 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:29.437574 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:29.437586 (+ 12us) server_negotiation.cc:407] Connection header received
0811 11:55:29.437638 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:29.437640 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:29.437690 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:29.437787 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:29.438288 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.439030 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.439815 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.440121 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.440240 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.441365 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:29.441369 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:29.441371 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:29.441426 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:29.442975 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:29.443428 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:29.443431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:29.443432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:29.443501 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:29.443812 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:29.443815 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:29.443816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:29.444042 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:29.444167 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:29.444448 (+ 281us) server_negotiation.cc:299] Negotiation successful
0811 11:55:29.444548 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
Aug 11 11:55:29 dist-test-slave-zkpd krb5kdc[8526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 11:55:29 dist-test-slave-zkpd krb5kdc[8526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:29.471783 8625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.462696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50498 (local address 127.6.143.190:45171)
0811 11:55:29.462850 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:29.462858 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:29.463232 (+ 374us) server_negotiation.cc:407] Connection header received
0811 11:55:29.463376 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:29.463379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:29.463476 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:29.463588 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:29.464508 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.465326 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.466403 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.466615 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.466701 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.469235 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:29.469238 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:29.469240 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:29.469265 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:29.470458 (+ 1193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:29.470826 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:29.470828 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:29.470829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:29.470903 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:29.471130 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:29.471133 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:29.471134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:29.471301 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:29.471407 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:29.471572 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 11:55:29.471662 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 11:55:29.474586 8572 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50498:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:55:29.477802 8572 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:55:29.493084 9116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.486400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:45063 (local address 127.0.0.1:41936)
0811 11:55:29.487661 (+ 1261us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:29.487715 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:29.487850 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:29.489161 (+ 1311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:29.489164 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:29.489179 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:29.489243 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.489246 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.490586 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.490590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.491633 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.491641 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.492513 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.492516 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.492641 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.492694 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 11:55:29.492785 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 11:55:29.492922 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":853,"spinlock_wait_cycles":3200,"thread_start_us":54,"threads_started":1}
I20260811 11:55:29.493088 9114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.486534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41936 (local address 127.6.143.129:45063)
0811 11:55:29.488732 (+ 2198us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:29.488738 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:29.488759 (+ 21us) server_negotiation.cc:407] Connection header received
0811 11:55:29.488833 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:29.488838 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:29.489027 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:29.489176 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:29.489366 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.490463 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.491750 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.492376 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.492517 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.492659 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:29.492842 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 11:55:29.492948 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2076,"thread_start_us":86,"threads_started":1}
I20260811 11:55:29.493767 9113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.487072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:43925 (local address 127.0.0.1:38508)
0811 11:55:29.489379 (+ 2307us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:29.489394 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:29.489485 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:29.490068 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:29.490071 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:29.490088 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:29.490170 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.490175 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.491204 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.491206 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.492323 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.492333 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.493375 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.493378 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.493459 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.493481 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 11:55:29.493543 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 11:55:29.493618 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2237,"thread_start_us":106,"threads_started":1}
I20260811 11:55:29.493827 9118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.487507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38508 (local address 127.6.143.131:43925)
0811 11:55:29.487822 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:29.487832 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:29.489495 (+ 1663us) server_negotiation.cc:407] Connection header received
0811 11:55:29.489579 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:29.489606 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:29.489782 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:29.489931 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:29.490288 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.491083 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.492450 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.493254 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.493371 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.493505 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:29.493571 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 11:55:29.493664 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":130,"threads_started":1}
I20260811 11:55:29.494431 9115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.486542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:36051 (local address 127.0.0.1:50544)
0811 11:55:29.487661 (+ 1119us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:29.487715 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:29.487850 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:29.488494 (+ 644us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:29.488499 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:29.488520 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:29.488615 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.488619 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.489989 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.489993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.491330 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.491341 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.494004 (+ 2663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.494007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.494112 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.494136 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 11:55:29.494219 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 11:55:29.494300 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1003,"spinlock_wait_cycles":6656,"thread_start_us":98,"threads_started":1}
I20260811 11:55:29.494635 9117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.487131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50544 (local address 127.6.143.130:36051)
0811 11:55:29.488097 (+ 966us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:29.488106 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:29.488125 (+ 19us) server_negotiation.cc:407] Connection header received
0811 11:55:29.488192 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:29.488195 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:29.488351 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:29.488479 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:29.488736 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.489862 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.491486 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.492324 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.492465 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.492621 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:29.494279 (+ 1658us) server_negotiation.cc:299] Negotiation successful
0811 11:55:29.494374 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":882,"thread_start_us":173,"threads_started":1}
I20260811 11:55:29.496354 8720 tablet_service.cc:1467] Processing CreateTablet for tablet 214adeb39379472ea74e4fcf556e62aa (DEFAULT_TABLE table=test-table [id=88359458fdf04f30914986c82ce66a57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:29.496685 9022 tablet_service.cc:1467] Processing CreateTablet for tablet 214adeb39379472ea74e4fcf556e62aa (DEFAULT_TABLE table=test-table [id=88359458fdf04f30914986c82ce66a57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:29.497526 8720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 214adeb39379472ea74e4fcf556e62aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:29.497654 9022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 214adeb39379472ea74e4fcf556e62aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:29.497905 8868 tablet_service.cc:1467] Processing CreateTablet for tablet 214adeb39379472ea74e4fcf556e62aa (DEFAULT_TABLE table=test-table [id=88359458fdf04f30914986c82ce66a57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:29.498845 8868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 214adeb39379472ea74e4fcf556e62aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:29.504465 9119 tablet_bootstrap.cc:492] T 214adeb39379472ea74e4fcf556e62aa P a4d81298eb6e46bc95c4ab437d740735: Bootstrap starting.
I20260811 11:55:29.506255 9121 tablet_bootstrap.cc:492] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7: Bootstrap starting.
I20260811 11:55:29.506695 9119 tablet_bootstrap.cc:654] T 214adeb39379472ea74e4fcf556e62aa P a4d81298eb6e46bc95c4ab437d740735: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:29.507242 9120 tablet_bootstrap.cc:492] T 214adeb39379472ea74e4fcf556e62aa P 166d007d7628419abf3b0f98ac59a04f: Bootstrap starting.
I20260811 11:55:29.507606 9119 log.cc:824] T 214adeb39379472ea74e4fcf556e62aa P a4d81298eb6e46bc95c4ab437d740735: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:29.508574 9121 tablet_bootstrap.cc:654] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:29.509363 9120 tablet_bootstrap.cc:654] T 214adeb39379472ea74e4fcf556e62aa P 166d007d7628419abf3b0f98ac59a04f: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:29.509377 9121 log.cc:824] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:29.509532 9119 tablet_bootstrap.cc:492] T 214adeb39379472ea74e4fcf556e62aa P a4d81298eb6e46bc95c4ab437d740735: No bootstrap required, opened a new log
I20260811 11:55:29.509768 9119 ts_tablet_manager.cc:1397] T 214adeb39379472ea74e4fcf556e62aa P a4d81298eb6e46bc95c4ab437d740735: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 11:55:29.510614 9120 log.cc:824] T 214adeb39379472ea74e4fcf556e62aa P 166d007d7628419abf3b0f98ac59a04f: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:29.511335 9121 tablet_bootstrap.cc:492] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7: No bootstrap required, opened a new log
I20260811 11:55:29.511546 9121 ts_tablet_manager.cc:1397] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 11:55:29.512506 9120 tablet_bootstrap.cc:492] T 214adeb39379472ea74e4fcf556e62aa P 166d007d7628419abf3b0f98ac59a04f: No bootstrap required, opened a new log
I20260811 11:55:29.512737 9120 ts_tablet_manager.cc:1397] T 214adeb39379472ea74e4fcf556e62aa P 166d007d7628419abf3b0f98ac59a04f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.001s
I20260811 11:55:29.513401 9119 raft_consensus.cc:348] T 214adeb39379472ea74e4fcf556e62aa P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:29.513633 9119 raft_consensus.cc:374] T 214adeb39379472ea74e4fcf556e62aa P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:29.513725 9119 raft_consensus.cc:729] T 214adeb39379472ea74e4fcf556e62aa P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4d81298eb6e46bc95c4ab437d740735, State: Initialized, Role: FOLLOWER
I20260811 11:55:29.514196 9119 consensus_queue.cc:260] T 214adeb39379472ea74e4fcf556e62aa P a4d81298eb6e46bc95c4ab437d740735 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:29.514824 9094 heartbeater.cc:503] Master 127.6.143.190:45171 was elected leader, sending a full tablet report...
I20260811 11:55:29.515195 9119 ts_tablet_manager.cc:1428] T 214adeb39379472ea74e4fcf556e62aa P a4d81298eb6e46bc95c4ab437d740735: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 11:55:29.515373 9121 raft_consensus.cc:348] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:29.515596 9121 raft_consensus.cc:374] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:29.515647 9121 raft_consensus.cc:729] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fed2e14718264870afb1a78b04a64ee7, State: Initialized, Role: FOLLOWER
I20260811 11:55:29.516111 9121 consensus_queue.cc:260] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:29.516588 9120 raft_consensus.cc:348] T 214adeb39379472ea74e4fcf556e62aa P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:29.516865 9120 raft_consensus.cc:374] T 214adeb39379472ea74e4fcf556e62aa P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:29.516852 8941 heartbeater.cc:503] Master 127.6.143.190:45171 was elected leader, sending a full tablet report...
I20260811 11:55:29.517000 9120 raft_consensus.cc:729] T 214adeb39379472ea74e4fcf556e62aa P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166d007d7628419abf3b0f98ac59a04f, State: Initialized, Role: FOLLOWER
I20260811 11:55:29.517155 9121 ts_tablet_manager.cc:1428] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 11:55:29.517477 9120 consensus_queue.cc:260] T 214adeb39379472ea74e4fcf556e62aa P 166d007d7628419abf3b0f98ac59a04f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:29.518076 8790 heartbeater.cc:503] Master 127.6.143.190:45171 was elected leader, sending a full tablet report...
I20260811 11:55:29.518384 9120 ts_tablet_manager.cc:1428] T 214adeb39379472ea74e4fcf556e62aa P 166d007d7628419abf3b0f98ac59a04f: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260811 11:55:29.565317 9127 raft_consensus.cc:482] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:55:29.565577 9127 raft_consensus.cc:504] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:29.566910 9127 leader_election.cc:288] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 166d007d7628419abf3b0f98ac59a04f (127.6.143.129:45063), a4d81298eb6e46bc95c4ab437d740735 (127.6.143.131:43925)
I20260811 11:55:29.570816 9114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.567231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:43951 (local address 127.6.143.129:45063)
0811 11:55:29.567361 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:29.567366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:29.567385 (+ 19us) server_negotiation.cc:407] Connection header received
0811 11:55:29.567598 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:29.567601 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:29.567666 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:29.567728 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:29.568134 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.568772 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.569844 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.570242 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.570387 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.570447 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:29.570626 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 11:55:29.570695 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 11:55:29.570816 8825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.567174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:45063 (local address 127.6.143.130:43951)
0811 11:55:29.567309 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:29.567328 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:29.567440 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:29.567771 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:29.567778 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:29.567808 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:29.567991 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.568001 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.568896 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.568899 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.569706 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.569716 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.570372 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.570376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.570486 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.570517 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 11:55:29.570610 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 11:55:29.570687 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 11:55:29.571564 9118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.567408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:59743 (local address 127.6.143.131:43925)
0811 11:55:29.567547 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:29.567555 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:29.567828 (+ 273us) server_negotiation.cc:407] Connection header received
0811 11:55:29.568069 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:29.568072 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:29.568135 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:29.568198 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:29.568468 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.569037 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.570490 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.570927 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.571063 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.571118 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:29.571412 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 11:55:29.571474 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 11:55:29.571589 9129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.567181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:43925 (local address 127.6.143.130:59743)
0811 11:55:29.567738 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:29.567752 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:29.567916 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:29.568213 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:29.568216 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:29.568236 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:29.568356 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.568362 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.569177 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.569182 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.570347 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.570358 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.571121 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.571127 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.571266 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.571289 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 11:55:29.571378 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 11:55:29.571447 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"mutex_wait_us":137,"thread_start_us":141,"threads_started":1}
I20260811 11:55:29.571530 8741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "214adeb39379472ea74e4fcf556e62aa" candidate_uuid: "fed2e14718264870afb1a78b04a64ee7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d007d7628419abf3b0f98ac59a04f" is_pre_election: true
I20260811 11:55:29.571930 8741 raft_consensus.cc:2454] T 214adeb39379472ea74e4fcf556e62aa P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fed2e14718264870afb1a78b04a64ee7 in term 0.
I20260811 11:55:29.572168 9043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "214adeb39379472ea74e4fcf556e62aa" candidate_uuid: "fed2e14718264870afb1a78b04a64ee7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4d81298eb6e46bc95c4ab437d740735" is_pre_election: true
I20260811 11:55:29.572472 9043 raft_consensus.cc:2454] T 214adeb39379472ea74e4fcf556e62aa P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fed2e14718264870afb1a78b04a64ee7 in term 0.
I20260811 11:55:29.572458 8818 leader_election.cc:302] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 166d007d7628419abf3b0f98ac59a04f, fed2e14718264870afb1a78b04a64ee7; no voters:
I20260811 11:55:29.572747 9127 raft_consensus.cc:2781] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:55:29.572870 9127 raft_consensus.cc:482] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:55:29.572938 9127 raft_consensus.cc:3037] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:29.574036 9127 raft_consensus.cc:504] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:29.574358 9127 leader_election.cc:288] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7 [CANDIDATE]: Term 1 election: Requested vote from peers 166d007d7628419abf3b0f98ac59a04f (127.6.143.129:45063), a4d81298eb6e46bc95c4ab437d740735 (127.6.143.131:43925)
I20260811 11:55:29.574679 9043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "214adeb39379472ea74e4fcf556e62aa" candidate_uuid: "fed2e14718264870afb1a78b04a64ee7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4d81298eb6e46bc95c4ab437d740735"
I20260811 11:55:29.574671 8741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "214adeb39379472ea74e4fcf556e62aa" candidate_uuid: "fed2e14718264870afb1a78b04a64ee7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d007d7628419abf3b0f98ac59a04f"
I20260811 11:55:29.574795 9043 raft_consensus.cc:3037] T 214adeb39379472ea74e4fcf556e62aa P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:29.574795 8741 raft_consensus.cc:3037] T 214adeb39379472ea74e4fcf556e62aa P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:29.576113 8741 raft_consensus.cc:2454] T 214adeb39379472ea74e4fcf556e62aa P 166d007d7628419abf3b0f98ac59a04f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fed2e14718264870afb1a78b04a64ee7 in term 1.
I20260811 11:55:29.576113 9043 raft_consensus.cc:2454] T 214adeb39379472ea74e4fcf556e62aa P a4d81298eb6e46bc95c4ab437d740735 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fed2e14718264870afb1a78b04a64ee7 in term 1.
I20260811 11:55:29.576429 8818 leader_election.cc:302] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 166d007d7628419abf3b0f98ac59a04f, fed2e14718264870afb1a78b04a64ee7; no voters:
I20260811 11:55:29.576593 9127 raft_consensus.cc:2781] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:29.576818 9127 raft_consensus.cc:686] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7 [term 1 LEADER]: Becoming Leader. State: Replica: fed2e14718264870afb1a78b04a64ee7, State: Running, Role: LEADER
I20260811 11:55:29.577224 9127 consensus_queue.cc:237] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:29.580615 8572 catalog_manager.cc:5495] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7 reported cstate change: term changed from 0 to 1, leader changed from <none> to fed2e14718264870afb1a78b04a64ee7 (127.6.143.130). New cstate: current_term: 1 leader_uuid: "fed2e14718264870afb1a78b04a64ee7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } health_report { overall_health: UNKNOWN } } }
Aug 11 11:55:29 dist-test-slave-zkpd krb5kdc[8526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449328, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:29.618162 9137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.609659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:45171 (local address 127.0.0.1:50514)
0811 11:55:29.609982 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:29.609995 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:29.610142 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:29.610410 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:29.610413 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:29.610835 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:29.610936 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.610942 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.611724 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.611728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.612293 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.612301 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.612742 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.612746 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.612828 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.613371 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:29.613383 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:29.614757 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:29.616425 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:29.616436 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:29.616443 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:29.616693 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:29.617018 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:29.617021 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:29.617023 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:29.617131 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:29.617527 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:29.617535 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:29.617647 (+ 112us) client_negotiation.cc:769] Sending connection context
0811 11:55:29.617760 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 11:55:29.617920 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":120,"threads_started":1}
I20260811 11:55:29.618162 8625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.609713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50514 (local address 127.6.143.190:45171)
0811 11:55:29.609874 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:29.609881 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:29.610083 (+ 202us) server_negotiation.cc:407] Connection header received
0811 11:55:29.610242 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:29.610246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:29.610292 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:29.610393 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:29.611060 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.611603 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.612418 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.612631 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.612742 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.614934 (+ 2192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:29.614938 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:29.614940 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:29.614983 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:29.616307 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:29.616794 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:29.616799 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:29.616801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:29.616878 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:29.617229 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:29.617234 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:29.617235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:29.617420 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:29.617527 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:29.617816 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 11:55:29.617992 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":65,"server-negotiator.queue_time_us":55}
I20260811 11:55:29.621984 8572 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:50514:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 11:55:29.622627 8572 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:55:29.627511 8868 tablet_service.cc:1467] Processing CreateTablet for tablet b234f0ea0acc4220aaf850551f7e6376 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1d3410067e994127af0127011256a52c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 11:55:29.627422 8720 tablet_service.cc:1467] Processing CreateTablet for tablet b234f0ea0acc4220aaf850551f7e6376 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1d3410067e994127af0127011256a52c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 11:55:29.628188 8868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b234f0ea0acc4220aaf850551f7e6376. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:29.628193 8720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b234f0ea0acc4220aaf850551f7e6376. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:29.627868 9022 tablet_service.cc:1467] Processing CreateTablet for tablet b234f0ea0acc4220aaf850551f7e6376 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=1d3410067e994127af0127011256a52c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 11:55:29.628495 9022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b234f0ea0acc4220aaf850551f7e6376. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:29.630551 9121 tablet_bootstrap.cc:492] T b234f0ea0acc4220aaf850551f7e6376 P fed2e14718264870afb1a78b04a64ee7: Bootstrap starting.
I20260811 11:55:29.631189 9119 tablet_bootstrap.cc:492] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735: Bootstrap starting.
I20260811 11:55:29.631536 9121 tablet_bootstrap.cc:654] T b234f0ea0acc4220aaf850551f7e6376 P fed2e14718264870afb1a78b04a64ee7: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:29.632040 9120 tablet_bootstrap.cc:492] T b234f0ea0acc4220aaf850551f7e6376 P 166d007d7628419abf3b0f98ac59a04f: Bootstrap starting.
I20260811 11:55:29.632251 9119 tablet_bootstrap.cc:654] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:29.633036 9120 tablet_bootstrap.cc:654] T b234f0ea0acc4220aaf850551f7e6376 P 166d007d7628419abf3b0f98ac59a04f: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:29.633152 9121 tablet_bootstrap.cc:492] T b234f0ea0acc4220aaf850551f7e6376 P fed2e14718264870afb1a78b04a64ee7: No bootstrap required, opened a new log
I20260811 11:55:29.633248 9121 ts_tablet_manager.cc:1397] T b234f0ea0acc4220aaf850551f7e6376 P fed2e14718264870afb1a78b04a64ee7: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.002s
I20260811 11:55:29.633533 9119 tablet_bootstrap.cc:492] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735: No bootstrap required, opened a new log
I20260811 11:55:29.633636 9119 ts_tablet_manager.cc:1397] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:29.633796 9121 raft_consensus.cc:348] T b234f0ea0acc4220aaf850551f7e6376 P fed2e14718264870afb1a78b04a64ee7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:29.633947 9121 raft_consensus.cc:374] T b234f0ea0acc4220aaf850551f7e6376 P fed2e14718264870afb1a78b04a64ee7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:29.633988 9121 raft_consensus.cc:729] T b234f0ea0acc4220aaf850551f7e6376 P fed2e14718264870afb1a78b04a64ee7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fed2e14718264870afb1a78b04a64ee7, State: Initialized, Role: FOLLOWER
I20260811 11:55:29.634116 9121 consensus_queue.cc:260] T b234f0ea0acc4220aaf850551f7e6376 P fed2e14718264870afb1a78b04a64ee7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:29.634126 9119 raft_consensus.cc:348] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:29.634249 9119 raft_consensus.cc:374] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:29.634296 9119 raft_consensus.cc:729] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4d81298eb6e46bc95c4ab437d740735, State: Initialized, Role: FOLLOWER
I20260811 11:55:29.634418 9121 ts_tablet_manager.cc:1428] T b234f0ea0acc4220aaf850551f7e6376 P fed2e14718264870afb1a78b04a64ee7: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 11:55:29.634473 9119 consensus_queue.cc:260] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:29.634649 9127 tablet_replica.cc:439] T b234f0ea0acc4220aaf850551f7e6376 P fed2e14718264870afb1a78b04a64ee7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } } }
I20260811 11:55:29.634781 9127 tablet_replica.cc:442] T b234f0ea0acc4220aaf850551f7e6376 P fed2e14718264870afb1a78b04a64ee7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:55:29.634773 9125 tablet_replica.cc:439] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } } }
I20260811 11:55:29.634922 9125 tablet_replica.cc:442] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:55:29.634766 9119 ts_tablet_manager.cc:1428] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:55:29.635761 9120 tablet_bootstrap.cc:492] T b234f0ea0acc4220aaf850551f7e6376 P 166d007d7628419abf3b0f98ac59a04f: No bootstrap required, opened a new log
I20260811 11:55:29.635857 9120 ts_tablet_manager.cc:1397] T b234f0ea0acc4220aaf850551f7e6376 P 166d007d7628419abf3b0f98ac59a04f: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260811 11:55:29.636409 9120 raft_consensus.cc:348] T b234f0ea0acc4220aaf850551f7e6376 P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:29.636535 9120 raft_consensus.cc:374] T b234f0ea0acc4220aaf850551f7e6376 P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:29.636575 9120 raft_consensus.cc:729] T b234f0ea0acc4220aaf850551f7e6376 P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166d007d7628419abf3b0f98ac59a04f, State: Initialized, Role: FOLLOWER
I20260811 11:55:29.636713 9120 consensus_queue.cc:260] T b234f0ea0acc4220aaf850551f7e6376 P 166d007d7628419abf3b0f98ac59a04f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:29.636989 9120 ts_tablet_manager.cc:1428] T b234f0ea0acc4220aaf850551f7e6376 P 166d007d7628419abf3b0f98ac59a04f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 11:55:29.637017 9128 tablet_replica.cc:439] T b234f0ea0acc4220aaf850551f7e6376 P 166d007d7628419abf3b0f98ac59a04f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } } }
I20260811 11:55:29.637153 9128 tablet_replica.cc:442] T b234f0ea0acc4220aaf850551f7e6376 P 166d007d7628419abf3b0f98ac59a04f: This TxnStatusTablet replica's current role is: FOLLOWER
W20260811 11:55:29.681849 9095 tablet.cc:2369] T 214adeb39379472ea74e4fcf556e62aa P a4d81298eb6e46bc95c4ab437d740735: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 11:55:29.714041 8942 tablet.cc:2369] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 11:55:29.733101 8791 tablet.cc:2369] T 214adeb39379472ea74e4fcf556e62aa P 166d007d7628419abf3b0f98ac59a04f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:55:29.757227 9125 raft_consensus.cc:482] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:55:29.757383 9125 raft_consensus.cc:504] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:29.758657 9125 leader_election.cc:288] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 166d007d7628419abf3b0f98ac59a04f (127.6.143.129:45063), fed2e14718264870afb1a78b04a64ee7 (127.6.143.130:36051)
I20260811 11:55:29.762997 8975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.759278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:45063 (local address 127.6.143.131:35797)
0811 11:55:29.759389 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:29.759404 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:29.759515 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:29.759819 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:29.759822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:29.759840 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:29.759997 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.760003 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.760824 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.760828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.761662 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.761671 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.762555 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.762559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.762675 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.762699 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 11:55:29.762782 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 11:55:29.762854 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 11:55:29.763168 9114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.758953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:35797 (local address 127.6.143.129:45063)
0811 11:55:29.759119 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:29.759141 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:29.759485 (+ 344us) server_negotiation.cc:407] Connection header received
0811 11:55:29.759636 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:29.759639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:29.759694 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:29.759806 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:29.760111 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.760701 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.761791 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.762319 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.762442 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.762494 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:29.762842 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 11:55:29.762983 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 11:55:29.763624 9141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.759289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:36051 (local address 127.6.143.131:59179)
0811 11:55:29.759670 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:29.759682 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:29.759865 (+ 183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:29.760149 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:29.760151 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:29.760167 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:29.760289 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.760294 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.761087 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.761093 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.761980 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.761993 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.762950 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.762953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.763259 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.763300 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 11:55:29.763407 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 11:55:29.763475 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":49,"thread_start_us":125,"threads_started":1}
I20260811 11:55:29.763667 9117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.759110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:59179 (local address 127.6.143.130:36051)
0811 11:55:29.759270 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:29.759274 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:29.759782 (+ 508us) server_negotiation.cc:407] Connection header received
0811 11:55:29.759998 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:29.760003 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:29.760051 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:29.760137 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:29.760397 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.760976 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.762123 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.762714 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.762818 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.762872 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:29.763429 (+ 557us) server_negotiation.cc:299] Negotiation successful
0811 11:55:29.763511 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 11:55:29.764314 8741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b234f0ea0acc4220aaf850551f7e6376" candidate_uuid: "a4d81298eb6e46bc95c4ab437d740735" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d007d7628419abf3b0f98ac59a04f" is_pre_election: true
I20260811 11:55:29.764338 8890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b234f0ea0acc4220aaf850551f7e6376" candidate_uuid: "a4d81298eb6e46bc95c4ab437d740735" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fed2e14718264870afb1a78b04a64ee7" is_pre_election: true
I20260811 11:55:29.764482 8741 raft_consensus.cc:2454] T b234f0ea0acc4220aaf850551f7e6376 P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4d81298eb6e46bc95c4ab437d740735 in term 0.
I20260811 11:55:29.764539 8890 raft_consensus.cc:2454] T b234f0ea0acc4220aaf850551f7e6376 P fed2e14718264870afb1a78b04a64ee7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4d81298eb6e46bc95c4ab437d740735 in term 0.
I20260811 11:55:29.764951 8968 leader_election.cc:302] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 166d007d7628419abf3b0f98ac59a04f, a4d81298eb6e46bc95c4ab437d740735, fed2e14718264870afb1a78b04a64ee7; no voters:
I20260811 11:55:29.765251 9125 raft_consensus.cc:2781] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:55:29.765360 9125 raft_consensus.cc:482] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:55:29.765408 9125 raft_consensus.cc:3037] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:29.766389 9125 raft_consensus.cc:504] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:29.766753 9125 leader_election.cc:288] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735 [CANDIDATE]: Term 1 election: Requested vote from peers 166d007d7628419abf3b0f98ac59a04f (127.6.143.129:45063), fed2e14718264870afb1a78b04a64ee7 (127.6.143.130:36051)
I20260811 11:55:29.767118 8741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b234f0ea0acc4220aaf850551f7e6376" candidate_uuid: "a4d81298eb6e46bc95c4ab437d740735" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d007d7628419abf3b0f98ac59a04f"
I20260811 11:55:29.767117 8890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b234f0ea0acc4220aaf850551f7e6376" candidate_uuid: "a4d81298eb6e46bc95c4ab437d740735" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fed2e14718264870afb1a78b04a64ee7"
I20260811 11:55:29.767236 8890 raft_consensus.cc:3037] T b234f0ea0acc4220aaf850551f7e6376 P fed2e14718264870afb1a78b04a64ee7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:29.767236 8741 raft_consensus.cc:3037] T b234f0ea0acc4220aaf850551f7e6376 P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:29.768455 8741 raft_consensus.cc:2454] T b234f0ea0acc4220aaf850551f7e6376 P 166d007d7628419abf3b0f98ac59a04f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4d81298eb6e46bc95c4ab437d740735 in term 1.
I20260811 11:55:29.768447 8890 raft_consensus.cc:2454] T b234f0ea0acc4220aaf850551f7e6376 P fed2e14718264870afb1a78b04a64ee7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4d81298eb6e46bc95c4ab437d740735 in term 1.
I20260811 11:55:29.768761 8969 leader_election.cc:302] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a4d81298eb6e46bc95c4ab437d740735, fed2e14718264870afb1a78b04a64ee7; no voters:
I20260811 11:55:29.768960 9125 raft_consensus.cc:2781] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:29.769188 9125 raft_consensus.cc:686] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735 [term 1 LEADER]: Becoming Leader. State: Replica: a4d81298eb6e46bc95c4ab437d740735, State: Running, Role: LEADER
I20260811 11:55:29.769562 9125 consensus_queue.cc:237] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:29.771903 9142 tablet_replica.cc:439] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735: TxnStatusTablet state changed. Reason: New leader a4d81298eb6e46bc95c4ab437d740735. Latest consensus state: current_term: 1 leader_uuid: "a4d81298eb6e46bc95c4ab437d740735" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } } }
I20260811 11:55:29.772019 9142 tablet_replica.cc:442] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735: This TxnStatusTablet replica's current role is: LEADER
I20260811 11:55:29.772305 9144 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 11:55:29.772531 9144 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 11:55:29.772923 8572 catalog_manager.cc:5495] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4d81298eb6e46bc95c4ab437d740735 (127.6.143.131). New cstate: current_term: 1 leader_uuid: "a4d81298eb6e46bc95c4ab437d740735" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } health_report { overall_health: HEALTHY } } }
I20260811 11:55:29.790705 9137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.786939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:43925 (local address 127.0.0.1:38520)
0811 11:55:29.787105 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:29.787122 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:29.787235 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:29.787601 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:29.787604 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:29.787612 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 11:55:29.787723 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.787728 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.788500 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.788504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.789092 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.789100 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.789560 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.789564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.789665 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.789709 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 11:55:29.790414 (+ 705us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 11:55:29.790431 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 11:55:29.790502 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 11:55:29.790586 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 11:55:29.790705 9118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.786997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38520 (local address 127.6.143.131:43925)
0811 11:55:29.787122 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:29.787127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:29.787196 (+ 69us) server_negotiation.cc:407] Connection header received
0811 11:55:29.787402 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:29.787405 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:29.787469 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:29.787572 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:55:29.787828 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.788375 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.789212 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.789439 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.789545 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.789860 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:55:29.790315 (+ 455us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:55:29.790436 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:29.790512 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 11:55:29.790559 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 11:55:29.796006 8890 raft_consensus.cc:1261] T b234f0ea0acc4220aaf850551f7e6376 P fed2e14718264870afb1a78b04a64ee7 [term 1 FOLLOWER]: Refusing update from remote peer a4d81298eb6e46bc95c4ab437d740735: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:29.796096 8741 raft_consensus.cc:1261] T b234f0ea0acc4220aaf850551f7e6376 P 166d007d7628419abf3b0f98ac59a04f [term 1 FOLLOWER]: Refusing update from remote peer a4d81298eb6e46bc95c4ab437d740735: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:29.796684 9142 consensus_queue.cc:1035] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:29.796877 9125 consensus_queue.cc:1035] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:29.800163 9127 tablet_replica.cc:439] T b234f0ea0acc4220aaf850551f7e6376 P fed2e14718264870afb1a78b04a64ee7: TxnStatusTablet state changed. Reason: New leader a4d81298eb6e46bc95c4ab437d740735. Latest consensus state: current_term: 1 leader_uuid: "a4d81298eb6e46bc95c4ab437d740735" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } } }
I20260811 11:55:29.800282 9127 tablet_replica.cc:442] T b234f0ea0acc4220aaf850551f7e6376 P fed2e14718264870afb1a78b04a64ee7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:55:29.800582 9128 tablet_replica.cc:439] T b234f0ea0acc4220aaf850551f7e6376 P 166d007d7628419abf3b0f98ac59a04f: TxnStatusTablet state changed. Reason: New leader a4d81298eb6e46bc95c4ab437d740735. Latest consensus state: current_term: 1 leader_uuid: "a4d81298eb6e46bc95c4ab437d740735" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } } }
I20260811 11:55:29.800693 9128 tablet_replica.cc:442] T b234f0ea0acc4220aaf850551f7e6376 P 166d007d7628419abf3b0f98ac59a04f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:55:29.802582 9142 tablet_replica.cc:439] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a4d81298eb6e46bc95c4ab437d740735" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } } }
I20260811 11:55:29.802706 9142 tablet_replica.cc:442] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735: This TxnStatusTablet replica's current role is: LEADER
I20260811 11:55:29.803086 9127 tablet_replica.cc:439] T b234f0ea0acc4220aaf850551f7e6376 P fed2e14718264870afb1a78b04a64ee7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a4d81298eb6e46bc95c4ab437d740735" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } } }
I20260811 11:55:29.803179 9127 tablet_replica.cc:442] T b234f0ea0acc4220aaf850551f7e6376 P fed2e14718264870afb1a78b04a64ee7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:55:29.803565 9147 mvcc.cc:205] Tried to move back new op lower bound from 7317296454837293056 to 7317296454738636800. Current Snapshot: MvccSnapshot[applied={T|T < 7317296454837293056}]
I20260811 11:55:29.804327 9149 mvcc.cc:205] Tried to move back new op lower bound from 7317296454837293056 to 7317296454738636800. Current Snapshot: MvccSnapshot[applied={T|T < 7317296454837293056}]
I20260811 11:55:29.809087 9142 tablet_replica.cc:439] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "a4d81298eb6e46bc95c4ab437d740735" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } } }
I20260811 11:55:29.809201 9142 tablet_replica.cc:442] T b234f0ea0acc4220aaf850551f7e6376 P a4d81298eb6e46bc95c4ab437d740735: This TxnStatusTablet replica's current role is: LEADER
I20260811 11:55:29.810355 9128 tablet_replica.cc:439] T b234f0ea0acc4220aaf850551f7e6376 P 166d007d7628419abf3b0f98ac59a04f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "a4d81298eb6e46bc95c4ab437d740735" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } } }
I20260811 11:55:29.810477 9128 tablet_replica.cc:442] T b234f0ea0acc4220aaf850551f7e6376 P 166d007d7628419abf3b0f98ac59a04f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:55:29.810993 9148 mvcc.cc:205] Tried to move back new op lower bound from 7317296454837293056 to 7317296454738636800. Current Snapshot: MvccSnapshot[applied={T|T < 7317296454837293056}]
I20260811 11:55:29.816699 9117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.812567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50552 (local address 127.6.143.130:36051)
0811 11:55:29.812684 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:29.812687 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:29.812699 (+ 12us) server_negotiation.cc:407] Connection header received
0811 11:55:29.812866 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:29.812868 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:29.812918 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:29.812999 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:55:29.813262 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.813839 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.815072 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.815422 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.815522 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.815769 (+ 247us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:55:29.816284 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:55:29.816398 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:29.816517 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 11:55:29.816572 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 11:55:29.827137 9118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.823844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:32991 (local address 127.6.143.131:43925)
0811 11:55:29.824011 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:29.824015 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:29.824174 (+ 159us) server_negotiation.cc:407] Connection header received
0811 11:55:29.824325 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:29.824328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:29.824383 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:29.824459 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:55:29.824720 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.825206 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.826058 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.826290 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.826404 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.826713 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:55:29.826787 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:55:29.826870 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:29.826963 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 11:55:29.827010 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 11:55:29.827162 9159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.823727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:43925 (local address 127.6.143.130:32991)
0811 11:55:29.824083 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:29.824096 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:29.824212 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:29.824480 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:29.824482 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:29.824490 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 11:55:29.824608 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.824613 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.825316 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.825322 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.825931 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.825939 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.826416 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.826420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.826513 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.826554 (+ 41us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 11:55:29.826880 (+ 326us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 11:55:29.826889 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 11:55:29.826952 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 11:55:29.827026 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":121,"threads_started":1}
I20260811 11:55:29.834270 8741 raft_consensus.cc:1261] T 214adeb39379472ea74e4fcf556e62aa P 166d007d7628419abf3b0f98ac59a04f [term 1 FOLLOWER]: Refusing update from remote peer fed2e14718264870afb1a78b04a64ee7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:29.834390 9043 raft_consensus.cc:1261] T 214adeb39379472ea74e4fcf556e62aa P a4d81298eb6e46bc95c4ab437d740735 [term 1 FOLLOWER]: Refusing update from remote peer fed2e14718264870afb1a78b04a64ee7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:29.834743 9127 consensus_queue.cc:1035] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:29.834983 9130 consensus_queue.cc:1035] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:29.857440 9101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.852080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:36051 (local address 127.6.143.131:44573)
0811 11:55:29.852260 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:29.852276 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:29.852364 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:29.852797 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:29.852799 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:29.852806 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 11:55:29.852903 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.852907 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.853839 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.853848 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.854650 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.854662 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.856665 (+ 2003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.856668 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.856767 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.856819 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 11:55:29.857161 (+ 342us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 11:55:29.857170 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 11:55:29.857237 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 11:55:29.857333 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 11:55:29.857432 9117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.852474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:44573 (local address 127.6.143.130:36051)
0811 11:55:29.852576 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:29.852580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:29.852594 (+ 14us) server_negotiation.cc:407] Connection header received
0811 11:55:29.852647 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:29.852649 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:29.852696 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:29.852778 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:55:29.853012 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.853704 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.855084 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.855382 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.855492 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.856970 (+ 1478us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:55:29.857046 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:55:29.857153 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:29.857249 (+ 96us) server_negotiation.cc:299] Negotiation successful
0811 11:55:29.857297 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 11:55:29.889145 8571 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:55:29.889343 8571 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:55:29.892179 8571 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 016ab2aae5fd45419161a6186a9719b6: Sending DeleteTablet for 3 replicas of tablet 214adeb39379472ea74e4fcf556e62aa
I20260811 11:55:29.892889 8720 tablet_service.cc:1514] Processing DeleteTablet for tablet 214adeb39379472ea74e4fcf556e62aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:29 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:41936
I20260811 11:55:29.893020 9022 tablet_service.cc:1514] Processing DeleteTablet for tablet 214adeb39379472ea74e4fcf556e62aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:29 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:38508
I20260811 11:55:29.893479 9168 tablet_replica.cc:330] T 214adeb39379472ea74e4fcf556e62aa P a4d81298eb6e46bc95c4ab437d740735: stopping tablet replica
I20260811 11:55:29.893779 9168 raft_consensus.cc:2229] T 214adeb39379472ea74e4fcf556e62aa P a4d81298eb6e46bc95c4ab437d740735 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:55:29.893958 9168 raft_consensus.cc:2258] T 214adeb39379472ea74e4fcf556e62aa P a4d81298eb6e46bc95c4ab437d740735 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:29.894663 9167 tablet_replica.cc:330] T 214adeb39379472ea74e4fcf556e62aa P 166d007d7628419abf3b0f98ac59a04f: stopping tablet replica
I20260811 11:55:29.894599 8571 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50498:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:55:29.895009 8571 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:55:29.895210 9167 raft_consensus.cc:2229] T 214adeb39379472ea74e4fcf556e62aa P 166d007d7628419abf3b0f98ac59a04f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:55:29.895491 9167 raft_consensus.cc:2258] T 214adeb39379472ea74e4fcf556e62aa P 166d007d7628419abf3b0f98ac59a04f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:29.895748 8868 tablet_service.cc:1514] Processing DeleteTablet for tablet 214adeb39379472ea74e4fcf556e62aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:29 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:50544
I20260811 11:55:29.896245 9169 tablet_replica.cc:330] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7: stopping tablet replica
I20260811 11:55:29.896569 9169 raft_consensus.cc:2229] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7 [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:55:29.897866 9167 ts_tablet_manager.cc:1905] T 214adeb39379472ea74e4fcf556e62aa P 166d007d7628419abf3b0f98ac59a04f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:29.901172 8720 tablet_service.cc:1467] Processing CreateTablet for tablet 597381db79584a5cb63bbcdf5c21b28d (DEFAULT_TABLE table=test-table [id=440fc4a0ed6c48baa4f45a0370ce42b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:29.901455 8720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 597381db79584a5cb63bbcdf5c21b28d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:29.902068 9169 raft_consensus.cc:2258] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:29.903127 9167 ts_tablet_manager.cc:1918] T 214adeb39379472ea74e4fcf556e62aa P 166d007d7628419abf3b0f98ac59a04f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 11:55:29.903271 9167 log.cc:1196] T 214adeb39379472ea74e4fcf556e62aa P 166d007d7628419abf3b0f98ac59a04f: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-0/wal/wals/214adeb39379472ea74e4fcf556e62aa
I20260811 11:55:29.903611 9167 ts_tablet_manager.cc:1939] T 214adeb39379472ea74e4fcf556e62aa P 166d007d7628419abf3b0f98ac59a04f: Deleting consensus metadata
I20260811 11:55:29.904000 9022 tablet_service.cc:1467] Processing CreateTablet for tablet 597381db79584a5cb63bbcdf5c21b28d (DEFAULT_TABLE table=test-table [id=440fc4a0ed6c48baa4f45a0370ce42b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:29.904299 9022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 597381db79584a5cb63bbcdf5c21b28d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:29.906750 9119 tablet_bootstrap.cc:492] T 597381db79584a5cb63bbcdf5c21b28d P a4d81298eb6e46bc95c4ab437d740735: Bootstrap starting.
I20260811 11:55:29.906914 9120 tablet_bootstrap.cc:492] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f: Bootstrap starting.
I20260811 11:55:29.907727 9119 tablet_bootstrap.cc:654] T 597381db79584a5cb63bbcdf5c21b28d P a4d81298eb6e46bc95c4ab437d740735: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:29.907821 9120 tablet_bootstrap.cc:654] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:29.908665 8556 catalog_manager.cc:4841] TS 166d007d7628419abf3b0f98ac59a04f (127.6.143.129:45063): tablet 214adeb39379472ea74e4fcf556e62aa (table test-table [id=88359458fdf04f30914986c82ce66a57]) successfully deleted
I20260811 11:55:29.909499 9169 ts_tablet_manager.cc:1905] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:29.909793 8868 tablet_service.cc:1467] Processing CreateTablet for tablet 597381db79584a5cb63bbcdf5c21b28d (DEFAULT_TABLE table=test-table [id=440fc4a0ed6c48baa4f45a0370ce42b3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:29.910060 8868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 597381db79584a5cb63bbcdf5c21b28d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:29.912492 9121 tablet_bootstrap.cc:492] T 597381db79584a5cb63bbcdf5c21b28d P fed2e14718264870afb1a78b04a64ee7: Bootstrap starting.
I20260811 11:55:29.912955 9169 ts_tablet_manager.cc:1918] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 11:55:29.913108 9169 log.cc:1196] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-1/wal/wals/214adeb39379472ea74e4fcf556e62aa
I20260811 11:55:29.913467 9169 ts_tablet_manager.cc:1939] T 214adeb39379472ea74e4fcf556e62aa P fed2e14718264870afb1a78b04a64ee7: Deleting consensus metadata
I20260811 11:55:29.913511 9121 tablet_bootstrap.cc:654] T 597381db79584a5cb63bbcdf5c21b28d P fed2e14718264870afb1a78b04a64ee7: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:29.913818 9168 ts_tablet_manager.cc:1905] T 214adeb39379472ea74e4fcf556e62aa P a4d81298eb6e46bc95c4ab437d740735: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:29.916000 8557 catalog_manager.cc:4841] TS fed2e14718264870afb1a78b04a64ee7 (127.6.143.130:36051): tablet 214adeb39379472ea74e4fcf556e62aa (table test-table [id=88359458fdf04f30914986c82ce66a57]) successfully deleted
I20260811 11:55:29.916859 9119 tablet_bootstrap.cc:492] T 597381db79584a5cb63bbcdf5c21b28d P a4d81298eb6e46bc95c4ab437d740735: No bootstrap required, opened a new log
I20260811 11:55:29.916939 9119 ts_tablet_manager.cc:1397] T 597381db79584a5cb63bbcdf5c21b28d P a4d81298eb6e46bc95c4ab437d740735: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260811 11:55:29.917083 9120 tablet_bootstrap.cc:492] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f: No bootstrap required, opened a new log
I20260811 11:55:29.917162 9120 ts_tablet_manager.cc:1397] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20260811 11:55:29.917362 9119 raft_consensus.cc:348] T 597381db79584a5cb63bbcdf5c21b28d P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:29.917465 9119 raft_consensus.cc:374] T 597381db79584a5cb63bbcdf5c21b28d P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:29.917495 9119 raft_consensus.cc:729] T 597381db79584a5cb63bbcdf5c21b28d P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4d81298eb6e46bc95c4ab437d740735, State: Initialized, Role: FOLLOWER
I20260811 11:55:29.917603 9119 consensus_queue.cc:260] T 597381db79584a5cb63bbcdf5c21b28d P a4d81298eb6e46bc95c4ab437d740735 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:29.917608 9120 raft_consensus.cc:348] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:29.917718 9120 raft_consensus.cc:374] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:29.917752 9120 raft_consensus.cc:729] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166d007d7628419abf3b0f98ac59a04f, State: Initialized, Role: FOLLOWER
I20260811 11:55:29.917872 9119 ts_tablet_manager.cc:1428] T 597381db79584a5cb63bbcdf5c21b28d P a4d81298eb6e46bc95c4ab437d740735: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:55:29.917881 9120 consensus_queue.cc:260] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:29.918150 9120 ts_tablet_manager.cc:1428] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260811 11:55:29.920570 9121 tablet_bootstrap.cc:492] T 597381db79584a5cb63bbcdf5c21b28d P fed2e14718264870afb1a78b04a64ee7: No bootstrap required, opened a new log
I20260811 11:55:29.920651 9121 ts_tablet_manager.cc:1397] T 597381db79584a5cb63bbcdf5c21b28d P fed2e14718264870afb1a78b04a64ee7: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s
I20260811 11:55:29.921142 9168 ts_tablet_manager.cc:1918] T 214adeb39379472ea74e4fcf556e62aa P a4d81298eb6e46bc95c4ab437d740735: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 11:55:29.921228 9168 log.cc:1196] T 214adeb39379472ea74e4fcf556e62aa P a4d81298eb6e46bc95c4ab437d740735: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-2/wal/wals/214adeb39379472ea74e4fcf556e62aa
I20260811 11:55:29.921175 9121 raft_consensus.cc:348] T 597381db79584a5cb63bbcdf5c21b28d P fed2e14718264870afb1a78b04a64ee7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:29.921283 9121 raft_consensus.cc:374] T 597381db79584a5cb63bbcdf5c21b28d P fed2e14718264870afb1a78b04a64ee7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:29.921334 9121 raft_consensus.cc:729] T 597381db79584a5cb63bbcdf5c21b28d P fed2e14718264870afb1a78b04a64ee7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fed2e14718264870afb1a78b04a64ee7, State: Initialized, Role: FOLLOWER
I20260811 11:55:29.921448 9121 consensus_queue.cc:260] T 597381db79584a5cb63bbcdf5c21b28d P fed2e14718264870afb1a78b04a64ee7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:29.921567 9168 ts_tablet_manager.cc:1939] T 214adeb39379472ea74e4fcf556e62aa P a4d81298eb6e46bc95c4ab437d740735: Deleting consensus metadata
I20260811 11:55:29.921703 9121 ts_tablet_manager.cc:1428] T 597381db79584a5cb63bbcdf5c21b28d P fed2e14718264870afb1a78b04a64ee7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:55:29.923607 8558 catalog_manager.cc:4841] TS a4d81298eb6e46bc95c4ab437d740735 (127.6.143.131:43925): tablet 214adeb39379472ea74e4fcf556e62aa (table test-table [id=88359458fdf04f30914986c82ce66a57]) successfully deleted
I20260811 11:55:29.925294 9128 raft_consensus.cc:482] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:55:29.925412 9128 raft_consensus.cc:504] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:29.926638 9128 leader_election.cc:288] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fed2e14718264870afb1a78b04a64ee7 (127.6.143.130:36051), a4d81298eb6e46bc95c4ab437d740735 (127.6.143.131:43925)
I20260811 11:55:29.932888 9118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.927255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:44093 (local address 127.6.143.131:43925)
0811 11:55:29.927368 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:29.927373 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:29.928211 (+ 838us) server_negotiation.cc:407] Connection header received
0811 11:55:29.928713 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:29.928716 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:29.928764 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:29.928813 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:29.929260 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.930004 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.931562 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.932193 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.932280 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.932330 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:29.932695 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 11:55:29.932765 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 11:55:29.933431 9174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.926875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:43925 (local address 127.6.143.129:44093)
0811 11:55:29.928120 (+ 1245us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:29.928136 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:29.928591 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:29.928977 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:29.928980 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:29.929000 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:29.929123 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.929130 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.930156 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.930159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.931414 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.931426 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.932440 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.932443 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.932540 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.932564 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 11:55:29.933206 (+ 642us) client_negotiation.cc:240] Negotiation successful
0811 11:55:29.933281 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1172,"thread_start_us":104,"threads_started":1}
I20260811 11:55:29.933486 9117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.927021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:55109 (local address 127.6.143.130:36051)
0811 11:55:29.927163 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:29.927169 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:29.928437 (+ 1268us) server_negotiation.cc:407] Connection header received
0811 11:55:29.928574 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:29.928577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:29.928624 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:29.928710 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:29.929095 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.929872 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.932220 (+ 2348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.932832 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.932919 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.932964 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:29.933310 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 11:55:29.933378 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 11:55:29.933851 9173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.926964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:36051 (local address 127.6.143.129:55109)
0811 11:55:29.928340 (+ 1376us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:29.928353 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:29.928462 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:29.928832 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:29.928835 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:29.928853 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:29.928965 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.928971 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.930016 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.930019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.932079 (+ 2060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.932090 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.933058 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.933061 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.933178 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.933199 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 11:55:29.933630 (+ 431us) client_negotiation.cc:240] Negotiation successful
0811 11:55:29.933723 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1317,"thread_start_us":90,"threads_started":1}
I20260811 11:55:29.934262 9043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "597381db79584a5cb63bbcdf5c21b28d" candidate_uuid: "166d007d7628419abf3b0f98ac59a04f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4d81298eb6e46bc95c4ab437d740735" is_pre_election: true
I20260811 11:55:29.934392 9043 raft_consensus.cc:2454] T 597381db79584a5cb63bbcdf5c21b28d P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 166d007d7628419abf3b0f98ac59a04f in term 0.
I20260811 11:55:29.934777 8890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "597381db79584a5cb63bbcdf5c21b28d" candidate_uuid: "166d007d7628419abf3b0f98ac59a04f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fed2e14718264870afb1a78b04a64ee7" is_pre_election: true
I20260811 11:55:29.934917 8890 raft_consensus.cc:2454] T 597381db79584a5cb63bbcdf5c21b28d P fed2e14718264870afb1a78b04a64ee7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 166d007d7628419abf3b0f98ac59a04f in term 0.
I20260811 11:55:29.935258 8672 leader_election.cc:302] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 166d007d7628419abf3b0f98ac59a04f, a4d81298eb6e46bc95c4ab437d740735; no voters:
I20260811 11:55:29.935519 9128 raft_consensus.cc:2781] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:55:29.935573 9128 raft_consensus.cc:482] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:55:29.935611 9128 raft_consensus.cc:3037] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:29.936877 9128 raft_consensus.cc:504] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:29.937218 9128 leader_election.cc:288] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f [CANDIDATE]: Term 1 election: Requested vote from peers fed2e14718264870afb1a78b04a64ee7 (127.6.143.130:36051), a4d81298eb6e46bc95c4ab437d740735 (127.6.143.131:43925)
I20260811 11:55:29.937630 8890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "597381db79584a5cb63bbcdf5c21b28d" candidate_uuid: "166d007d7628419abf3b0f98ac59a04f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fed2e14718264870afb1a78b04a64ee7"
I20260811 11:55:29.937758 8890 raft_consensus.cc:3037] T 597381db79584a5cb63bbcdf5c21b28d P fed2e14718264870afb1a78b04a64ee7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:29.938066 9043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "597381db79584a5cb63bbcdf5c21b28d" candidate_uuid: "166d007d7628419abf3b0f98ac59a04f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4d81298eb6e46bc95c4ab437d740735"
I20260811 11:55:29.938169 9043 raft_consensus.cc:3037] T 597381db79584a5cb63bbcdf5c21b28d P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:29.938874 8890 raft_consensus.cc:2454] T 597381db79584a5cb63bbcdf5c21b28d P fed2e14718264870afb1a78b04a64ee7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 166d007d7628419abf3b0f98ac59a04f in term 1.
I20260811 11:55:29.939177 9043 raft_consensus.cc:2454] T 597381db79584a5cb63bbcdf5c21b28d P a4d81298eb6e46bc95c4ab437d740735 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 166d007d7628419abf3b0f98ac59a04f in term 1.
I20260811 11:55:29.939208 8671 leader_election.cc:302] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 166d007d7628419abf3b0f98ac59a04f, fed2e14718264870afb1a78b04a64ee7; no voters:
I20260811 11:55:29.939633 9128 raft_consensus.cc:2781] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:29.939833 9128 raft_consensus.cc:686] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f [term 1 LEADER]: Becoming Leader. State: Replica: 166d007d7628419abf3b0f98ac59a04f, State: Running, Role: LEADER
I20260811 11:55:29.940088 9128 consensus_queue.cc:237] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:29.942301 8572 catalog_manager.cc:5495] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f reported cstate change: term changed from 0 to 1, leader changed from <none> to 166d007d7628419abf3b0f98ac59a04f (127.6.143.129). New cstate: current_term: 1 leader_uuid: "166d007d7628419abf3b0f98ac59a04f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } health_report { overall_health: UNKNOWN } } }
I20260811 11:55:29.966144 9114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.961882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41948 (local address 127.6.143.129:45063)
0811 11:55:29.961981 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:29.961986 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:29.962185 (+ 199us) server_negotiation.cc:407] Connection header received
0811 11:55:29.962260 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:29.962263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:29.962322 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:29.962414 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:55:29.962763 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.963506 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.964664 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.964889 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.965001 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.965300 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:55:29.965729 (+ 429us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:55:29.965861 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:29.965941 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 11:55:29.965999 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260811 11:55:29.977321 9118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.973913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:59277 (local address 127.6.143.131:43925)
0811 11:55:29.974073 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:29.974076 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:29.974173 (+ 97us) server_negotiation.cc:407] Connection header received
0811 11:55:29.974295 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:29.974297 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:29.974338 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:29.974401 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:55:29.974660 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.975119 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.976352 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.976561 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.976636 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.976953 (+ 317us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:55:29.977012 (+ 59us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:55:29.977065 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:29.977185 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 11:55:29.977227 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 11:55:29.977414 9180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.973756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:43925 (local address 127.6.143.129:59277)
0811 11:55:29.974077 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:29.974091 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:29.974202 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:29.974431 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:29.974434 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:29.974442 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 11:55:29.974558 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.974563 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.975218 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.975222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.976244 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.976255 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:29.976665 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:29.976668 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:29.976781 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:29.976828 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 11:55:29.977094 (+ 266us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 11:55:29.977101 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 11:55:29.977168 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 11:55:29.977250 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":132,"threads_started":1}
I20260811 11:55:29.983203 8890 raft_consensus.cc:1261] T 597381db79584a5cb63bbcdf5c21b28d P fed2e14718264870afb1a78b04a64ee7 [term 1 FOLLOWER]: Refusing update from remote peer 166d007d7628419abf3b0f98ac59a04f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:29.983593 9043 raft_consensus.cc:1261] T 597381db79584a5cb63bbcdf5c21b28d P a4d81298eb6e46bc95c4ab437d740735 [term 1 FOLLOWER]: Refusing update from remote peer 166d007d7628419abf3b0f98ac59a04f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:29.983635 9175 consensus_queue.cc:1035] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:29.983954 9128 consensus_queue.cc:1035] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:30.004166 9114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.998736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:45483 (local address 127.6.143.129:45063)
0811 11:55:29.998872 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:29.998876 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:29.998887 (+ 11us) server_negotiation.cc:407] Connection header received
0811 11:55:29.998925 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:29.998927 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:29.998985 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:29.999050 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:55:30.000342 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:30.001060 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:30.002820 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:30.003157 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:30.003297 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:30.003545 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:55:30.003624 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:55:30.003694 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:30.003960 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 11:55:30.004011 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67,"spinlock_wait_cycles":23424}
I20260811 11:55:30.004449 9101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:29.998178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:45063 (local address 127.6.143.131:45483)
0811 11:55:29.998533 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:29.998548 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:29.998620 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:29.999712 (+ 1092us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:29.999715 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:29.999724 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 11:55:29.999815 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:29.999820 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:30.001721 (+ 1901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:30.001724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:30.002634 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:30.002645 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:30.003279 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:30.003283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:30.003373 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:30.003409 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 11:55:30.003738 (+ 329us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 11:55:30.003745 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 11:55:30.003807 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 11:55:30.004318 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275}
W20260811 11:55:30.005193 9148 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 11:55:31.562851 8571 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:55:31.562997 8571 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50498:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:55:31.565294 8571 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 016ab2aae5fd45419161a6186a9719b6: Sending DeleteTablet for 3 replicas of tablet 597381db79584a5cb63bbcdf5c21b28d
I20260811 11:55:31.565810 8720 tablet_service.cc:1514] Processing DeleteTablet for tablet 597381db79584a5cb63bbcdf5c21b28d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:31 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:41936
I20260811 11:55:31.565847 8868 tablet_service.cc:1514] Processing DeleteTablet for tablet 597381db79584a5cb63bbcdf5c21b28d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:31 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:50544
I20260811 11:55:31.566022 9022 tablet_service.cc:1514] Processing DeleteTablet for tablet 597381db79584a5cb63bbcdf5c21b28d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:31 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:38508
I20260811 11:55:31.566210 9212 tablet_replica.cc:330] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f: stopping tablet replica
I20260811 11:55:31.566339 9212 raft_consensus.cc:2229] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:55:31.566666 9212 raft_consensus.cc:2258] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:31.567310 9212 ts_tablet_manager.cc:1905] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:31.569873 9213 tablet_replica.cc:330] T 597381db79584a5cb63bbcdf5c21b28d P fed2e14718264870afb1a78b04a64ee7: stopping tablet replica
I20260811 11:55:31.570027 9213 raft_consensus.cc:2229] T 597381db79584a5cb63bbcdf5c21b28d P fed2e14718264870afb1a78b04a64ee7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:55:31.570137 9213 raft_consensus.cc:2258] T 597381db79584a5cb63bbcdf5c21b28d P fed2e14718264870afb1a78b04a64ee7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:31.570369 9214 tablet_replica.cc:330] T 597381db79584a5cb63bbcdf5c21b28d P a4d81298eb6e46bc95c4ab437d740735: stopping tablet replica
I20260811 11:55:31.570528 9214 raft_consensus.cc:2229] T 597381db79584a5cb63bbcdf5c21b28d P a4d81298eb6e46bc95c4ab437d740735 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:55:31.570609 9212 ts_tablet_manager.cc:1918] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 11:55:31.570650 9214 raft_consensus.cc:2258] T 597381db79584a5cb63bbcdf5c21b28d P a4d81298eb6e46bc95c4ab437d740735 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:31.570684 9212 log.cc:1196] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-0/wal/wals/597381db79584a5cb63bbcdf5c21b28d
I20260811 11:55:31.571134 9212 ts_tablet_manager.cc:1939] T 597381db79584a5cb63bbcdf5c21b28d P 166d007d7628419abf3b0f98ac59a04f: Deleting consensus metadata
I20260811 11:55:31.571363 9213 ts_tablet_manager.cc:1905] T 597381db79584a5cb63bbcdf5c21b28d P fed2e14718264870afb1a78b04a64ee7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:31.571826 8556 catalog_manager.cc:4841] TS 166d007d7628419abf3b0f98ac59a04f (127.6.143.129:45063): tablet 597381db79584a5cb63bbcdf5c21b28d (table test-table [id=440fc4a0ed6c48baa4f45a0370ce42b3]) successfully deleted
I20260811 11:55:31.574702 9213 ts_tablet_manager.cc:1918] T 597381db79584a5cb63bbcdf5c21b28d P fed2e14718264870afb1a78b04a64ee7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 11:55:31.574783 9213 log.cc:1196] T 597381db79584a5cb63bbcdf5c21b28d P fed2e14718264870afb1a78b04a64ee7: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-1/wal/wals/597381db79584a5cb63bbcdf5c21b28d
I20260811 11:55:31.575026 9214 ts_tablet_manager.cc:1905] T 597381db79584a5cb63bbcdf5c21b28d P a4d81298eb6e46bc95c4ab437d740735: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:31.575064 9213 ts_tablet_manager.cc:1939] T 597381db79584a5cb63bbcdf5c21b28d P fed2e14718264870afb1a78b04a64ee7: Deleting consensus metadata
I20260811 11:55:31.575687 8557 catalog_manager.cc:4841] TS fed2e14718264870afb1a78b04a64ee7 (127.6.143.130:36051): tablet 597381db79584a5cb63bbcdf5c21b28d (table test-table [id=440fc4a0ed6c48baa4f45a0370ce42b3]) successfully deleted
I20260811 11:55:31.577664 9214 ts_tablet_manager.cc:1918] T 597381db79584a5cb63bbcdf5c21b28d P a4d81298eb6e46bc95c4ab437d740735: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 11:55:31.577729 9214 log.cc:1196] T 597381db79584a5cb63bbcdf5c21b28d P a4d81298eb6e46bc95c4ab437d740735: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-2/wal/wals/597381db79584a5cb63bbcdf5c21b28d
I20260811 11:55:31.577976 9214 ts_tablet_manager.cc:1939] T 597381db79584a5cb63bbcdf5c21b28d P a4d81298eb6e46bc95c4ab437d740735: Deleting consensus metadata
I20260811 11:55:31.578598 8558 catalog_manager.cc:4841] TS a4d81298eb6e46bc95c4ab437d740735 (127.6.143.131:43925): tablet 597381db79584a5cb63bbcdf5c21b28d (table test-table [id=440fc4a0ed6c48baa4f45a0370ce42b3]) successfully deleted
Aug 11 11:55:31 dist-test-slave-zkpd krb5kdc[8526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449331, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 11:55:31 dist-test-slave-zkpd krb5kdc[8526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449331, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:31.596246 9223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:31.588124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56586 (local address 127.6.143.190:45171)
0811 11:55:31.588416 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:31.588423 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:31.588437 (+ 14us) server_negotiation.cc:407] Connection header received
0811 11:55:31.588571 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:31.588573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:31.588615 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:31.588724 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:31.589393 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:31.589994 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:31.590807 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:31.591030 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:31.591111 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:31.593385 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:31.593388 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:31.593390 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:31.593412 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:31.594709 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:31.595085 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:31.595087 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:31.595088 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:31.595162 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:31.595396 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:31.595399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:31.595400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:31.595558 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:31.595657 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:31.595850 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 11:55:31.596026 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":95,"threads_started":1}
I20260811 11:55:31.598560 8571 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56586:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:55:31.599004 8571 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:55:31.603583 8720 tablet_service.cc:1467] Processing CreateTablet for tablet 4b316a2376a94d32a93798165b0f402c (DEFAULT_TABLE table=test-table [id=a5cadfd8cb524f54aaa74b2150f36a38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:31.603914 8720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b316a2376a94d32a93798165b0f402c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:31.603807 8868 tablet_service.cc:1467] Processing CreateTablet for tablet 4b316a2376a94d32a93798165b0f402c (DEFAULT_TABLE table=test-table [id=a5cadfd8cb524f54aaa74b2150f36a38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:31.603806 9022 tablet_service.cc:1467] Processing CreateTablet for tablet 4b316a2376a94d32a93798165b0f402c (DEFAULT_TABLE table=test-table [id=a5cadfd8cb524f54aaa74b2150f36a38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:31.604141 8868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b316a2376a94d32a93798165b0f402c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:31.604151 9022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4b316a2376a94d32a93798165b0f402c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:31.605950 9224 tablet_bootstrap.cc:492] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735: Bootstrap starting.
I20260811 11:55:31.606521 9225 tablet_bootstrap.cc:492] T 4b316a2376a94d32a93798165b0f402c P 166d007d7628419abf3b0f98ac59a04f: Bootstrap starting.
I20260811 11:55:31.606806 9226 tablet_bootstrap.cc:492] T 4b316a2376a94d32a93798165b0f402c P fed2e14718264870afb1a78b04a64ee7: Bootstrap starting.
I20260811 11:55:31.606894 9224 tablet_bootstrap.cc:654] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:31.607443 9225 tablet_bootstrap.cc:654] T 4b316a2376a94d32a93798165b0f402c P 166d007d7628419abf3b0f98ac59a04f: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:31.607748 9226 tablet_bootstrap.cc:654] T 4b316a2376a94d32a93798165b0f402c P fed2e14718264870afb1a78b04a64ee7: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:31.608170 9224 tablet_bootstrap.cc:492] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735: No bootstrap required, opened a new log
I20260811 11:55:31.608271 9224 ts_tablet_manager.cc:1397] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:31.608596 9225 tablet_bootstrap.cc:492] T 4b316a2376a94d32a93798165b0f402c P 166d007d7628419abf3b0f98ac59a04f: No bootstrap required, opened a new log
I20260811 11:55:31.608667 9225 ts_tablet_manager.cc:1397] T 4b316a2376a94d32a93798165b0f402c P 166d007d7628419abf3b0f98ac59a04f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260811 11:55:31.608618 9224 raft_consensus.cc:348] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:31.608750 9224 raft_consensus.cc:374] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:31.608795 9224 raft_consensus.cc:729] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4d81298eb6e46bc95c4ab437d740735, State: Initialized, Role: FOLLOWER
I20260811 11:55:31.608922 9224 consensus_queue.cc:260] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:31.609153 9225 raft_consensus.cc:348] T 4b316a2376a94d32a93798165b0f402c P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:31.609263 9225 raft_consensus.cc:374] T 4b316a2376a94d32a93798165b0f402c P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:31.609289 9224 ts_tablet_manager.cc:1428] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:55:31.609297 9225 raft_consensus.cc:729] T 4b316a2376a94d32a93798165b0f402c P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166d007d7628419abf3b0f98ac59a04f, State: Initialized, Role: FOLLOWER
I20260811 11:55:31.609452 9225 consensus_queue.cc:260] T 4b316a2376a94d32a93798165b0f402c P 166d007d7628419abf3b0f98ac59a04f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:31.609728 9225 ts_tablet_manager.cc:1428] T 4b316a2376a94d32a93798165b0f402c P 166d007d7628419abf3b0f98ac59a04f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 11:55:31.610525 9226 tablet_bootstrap.cc:492] T 4b316a2376a94d32a93798165b0f402c P fed2e14718264870afb1a78b04a64ee7: No bootstrap required, opened a new log
I20260811 11:55:31.610633 9226 ts_tablet_manager.cc:1397] T 4b316a2376a94d32a93798165b0f402c P fed2e14718264870afb1a78b04a64ee7: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 11:55:31.610991 9226 raft_consensus.cc:348] T 4b316a2376a94d32a93798165b0f402c P fed2e14718264870afb1a78b04a64ee7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:31.611104 9226 raft_consensus.cc:374] T 4b316a2376a94d32a93798165b0f402c P fed2e14718264870afb1a78b04a64ee7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:31.611145 9226 raft_consensus.cc:729] T 4b316a2376a94d32a93798165b0f402c P fed2e14718264870afb1a78b04a64ee7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fed2e14718264870afb1a78b04a64ee7, State: Initialized, Role: FOLLOWER
I20260811 11:55:31.611284 9226 consensus_queue.cc:260] T 4b316a2376a94d32a93798165b0f402c P fed2e14718264870afb1a78b04a64ee7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:31.611685 9226 ts_tablet_manager.cc:1428] T 4b316a2376a94d32a93798165b0f402c P fed2e14718264870afb1a78b04a64ee7: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:55:31.637218 9204 raft_consensus.cc:482] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:55:31.637339 9204 raft_consensus.cc:504] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:31.637799 9204 leader_election.cc:288] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 166d007d7628419abf3b0f98ac59a04f (127.6.143.129:45063), fed2e14718264870afb1a78b04a64ee7 (127.6.143.130:36051)
I20260811 11:55:31.638187 8741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b316a2376a94d32a93798165b0f402c" candidate_uuid: "a4d81298eb6e46bc95c4ab437d740735" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d007d7628419abf3b0f98ac59a04f" is_pre_election: true
I20260811 11:55:31.638276 8890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b316a2376a94d32a93798165b0f402c" candidate_uuid: "a4d81298eb6e46bc95c4ab437d740735" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fed2e14718264870afb1a78b04a64ee7" is_pre_election: true
I20260811 11:55:31.638406 8741 raft_consensus.cc:2454] T 4b316a2376a94d32a93798165b0f402c P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4d81298eb6e46bc95c4ab437d740735 in term 0.
I20260811 11:55:31.638490 8890 raft_consensus.cc:2454] T 4b316a2376a94d32a93798165b0f402c P fed2e14718264870afb1a78b04a64ee7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a4d81298eb6e46bc95c4ab437d740735 in term 0.
I20260811 11:55:31.638705 8968 leader_election.cc:302] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 166d007d7628419abf3b0f98ac59a04f, a4d81298eb6e46bc95c4ab437d740735; no voters:
I20260811 11:55:31.638890 9204 raft_consensus.cc:2781] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:55:31.638952 9204 raft_consensus.cc:482] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:55:31.638993 9204 raft_consensus.cc:3037] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:31.640051 9204 raft_consensus.cc:504] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:31.640419 9204 leader_election.cc:288] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735 [CANDIDATE]: Term 1 election: Requested vote from peers 166d007d7628419abf3b0f98ac59a04f (127.6.143.129:45063), fed2e14718264870afb1a78b04a64ee7 (127.6.143.130:36051)
I20260811 11:55:31.640733 8741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b316a2376a94d32a93798165b0f402c" candidate_uuid: "a4d81298eb6e46bc95c4ab437d740735" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "166d007d7628419abf3b0f98ac59a04f"
I20260811 11:55:31.640867 8741 raft_consensus.cc:3037] T 4b316a2376a94d32a93798165b0f402c P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:31.640812 8890 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4b316a2376a94d32a93798165b0f402c" candidate_uuid: "a4d81298eb6e46bc95c4ab437d740735" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fed2e14718264870afb1a78b04a64ee7"
I20260811 11:55:31.640960 8890 raft_consensus.cc:3037] T 4b316a2376a94d32a93798165b0f402c P fed2e14718264870afb1a78b04a64ee7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:31.641767 8741 raft_consensus.cc:2454] T 4b316a2376a94d32a93798165b0f402c P 166d007d7628419abf3b0f98ac59a04f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4d81298eb6e46bc95c4ab437d740735 in term 1.
I20260811 11:55:31.641932 8890 raft_consensus.cc:2454] T 4b316a2376a94d32a93798165b0f402c P fed2e14718264870afb1a78b04a64ee7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a4d81298eb6e46bc95c4ab437d740735 in term 1.
I20260811 11:55:31.642097 8968 leader_election.cc:302] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 166d007d7628419abf3b0f98ac59a04f, a4d81298eb6e46bc95c4ab437d740735; no voters:
I20260811 11:55:31.642261 9204 raft_consensus.cc:2781] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:31.642355 9204 raft_consensus.cc:686] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735 [term 1 LEADER]: Becoming Leader. State: Replica: a4d81298eb6e46bc95c4ab437d740735, State: Running, Role: LEADER
I20260811 11:55:31.642495 9204 consensus_queue.cc:237] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:31.644114 8571 catalog_manager.cc:5495] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735 reported cstate change: term changed from 0 to 1, leader changed from <none> to a4d81298eb6e46bc95c4ab437d740735 (127.6.143.131). New cstate: current_term: 1 leader_uuid: "a4d81298eb6e46bc95c4ab437d740735" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } health_report { overall_health: HEALTHY } } }
I20260811 11:55:31.667510 9233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:31.664454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33492 (local address 127.6.143.131:43925)
0811 11:55:31.664724 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:31.664731 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:31.664743 (+ 12us) server_negotiation.cc:407] Connection header received
0811 11:55:31.664782 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:31.664784 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:31.664825 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:31.664910 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:55:31.665142 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:31.665653 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:31.666454 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:31.666686 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:31.666760 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:31.667075 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:55:31.667145 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:55:31.667202 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:31.667306 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 11:55:31.667348 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":117,"threads_started":1}
I20260811 11:55:31.676257 9233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:31.673295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:40307 (local address 127.6.143.131:43925)
0811 11:55:31.673409 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:31.673413 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:31.673519 (+ 106us) server_negotiation.cc:407] Connection header received
0811 11:55:31.673660 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:31.673663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:31.673714 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:31.673790 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:55:31.673997 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:31.674459 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:31.675252 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:31.675450 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:31.675547 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:31.675780 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:55:31.675835 (+ 55us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:55:31.675948 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:31.676020 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 11:55:31.676109 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":43}
I20260811 11:55:31.676257 9236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:31.673131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:43925 (local address 127.6.143.131:40307)
0811 11:55:31.673420 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:31.673435 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:31.673560 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:31.673800 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:31.673802 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:31.673809 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 11:55:31.673887 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:31.673892 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:31.674561 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:31.674565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:31.675128 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:31.675136 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:31.675557 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:31.675561 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:31.675637 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:31.675667 (+ 30us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 11:55:31.675941 (+ 274us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 11:55:31.675947 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 11:55:31.676014 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 11:55:31.676120 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":2,"thread_start_us":120,"threads_started":1}
I20260811 11:55:31.681577 8890 raft_consensus.cc:1261] T 4b316a2376a94d32a93798165b0f402c P fed2e14718264870afb1a78b04a64ee7 [term 1 FOLLOWER]: Refusing update from remote peer a4d81298eb6e46bc95c4ab437d740735: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:31.681592 8741 raft_consensus.cc:1261] T 4b316a2376a94d32a93798165b0f402c P 166d007d7628419abf3b0f98ac59a04f [term 1 FOLLOWER]: Refusing update from remote peer a4d81298eb6e46bc95c4ab437d740735: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:31.682024 9125 consensus_queue.cc:1035] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735 [LEADER]: Connected to new peer: Peer: permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:31.682180 9204 consensus_queue.cc:1035] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:31.716234 8571 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:55:31.716423 8571 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:55:31.718644 8571 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 016ab2aae5fd45419161a6186a9719b6: Sending DeleteTablet for 3 replicas of tablet 4b316a2376a94d32a93798165b0f402c
I20260811 11:55:31.719177 8720 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b316a2376a94d32a93798165b0f402c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:31 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:41936
I20260811 11:55:31.719273 9001 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b316a2376a94d32a93798165b0f402c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:31 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:38508
I20260811 11:55:31.719273 8868 tablet_service.cc:1514] Processing DeleteTablet for tablet 4b316a2376a94d32a93798165b0f402c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:31 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:50544
I20260811 11:55:31.719455 9214 tablet_replica.cc:330] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735: stopping tablet replica
I20260811 11:55:31.719462 9212 tablet_replica.cc:330] T 4b316a2376a94d32a93798165b0f402c P 166d007d7628419abf3b0f98ac59a04f: stopping tablet replica
I20260811 11:55:31.719574 9212 raft_consensus.cc:2229] T 4b316a2376a94d32a93798165b0f402c P 166d007d7628419abf3b0f98ac59a04f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:55:31.719581 9214 raft_consensus.cc:2229] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735 [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:55:31.719677 9212 raft_consensus.cc:2258] T 4b316a2376a94d32a93798165b0f402c P 166d007d7628419abf3b0f98ac59a04f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:31.719842 9213 tablet_replica.cc:330] T 4b316a2376a94d32a93798165b0f402c P fed2e14718264870afb1a78b04a64ee7: stopping tablet replica
I20260811 11:55:31.719843 9214 raft_consensus.cc:2258] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:31.719972 9213 raft_consensus.cc:2229] T 4b316a2376a94d32a93798165b0f402c P fed2e14718264870afb1a78b04a64ee7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:55:31.720068 9213 raft_consensus.cc:2258] T 4b316a2376a94d32a93798165b0f402c P fed2e14718264870afb1a78b04a64ee7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:31.720472 8571 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56586:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 11:55:31.720867 9214 ts_tablet_manager.cc:1905] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:31.720945 9213 ts_tablet_manager.cc:1905] T 4b316a2376a94d32a93798165b0f402c P fed2e14718264870afb1a78b04a64ee7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 11:55:31.721131 8571 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:55:31.723276 9213 ts_tablet_manager.cc:1918] T 4b316a2376a94d32a93798165b0f402c P fed2e14718264870afb1a78b04a64ee7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 11:55:31.723378 9213 log.cc:1196] T 4b316a2376a94d32a93798165b0f402c P fed2e14718264870afb1a78b04a64ee7: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-1/wal/wals/4b316a2376a94d32a93798165b0f402c
I20260811 11:55:31.723470 9214 ts_tablet_manager.cc:1918] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 11:55:31.723546 9214 log.cc:1196] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-2/wal/wals/4b316a2376a94d32a93798165b0f402c
I20260811 11:55:31.723672 9213 ts_tablet_manager.cc:1939] T 4b316a2376a94d32a93798165b0f402c P fed2e14718264870afb1a78b04a64ee7: Deleting consensus metadata
I20260811 11:55:31.723835 9214 ts_tablet_manager.cc:1939] T 4b316a2376a94d32a93798165b0f402c P a4d81298eb6e46bc95c4ab437d740735: Deleting consensus metadata
I20260811 11:55:31.724330 8557 catalog_manager.cc:4841] TS fed2e14718264870afb1a78b04a64ee7 (127.6.143.130:36051): tablet 4b316a2376a94d32a93798165b0f402c (table test-table [id=a5cadfd8cb524f54aaa74b2150f36a38]) successfully deleted
I20260811 11:55:31.725438 8558 catalog_manager.cc:4841] TS a4d81298eb6e46bc95c4ab437d740735 (127.6.143.131:43925): tablet 4b316a2376a94d32a93798165b0f402c (table test-table [id=a5cadfd8cb524f54aaa74b2150f36a38]) successfully deleted
I20260811 11:55:31.725893 9212 ts_tablet_manager.cc:1905] T 4b316a2376a94d32a93798165b0f402c P 166d007d7628419abf3b0f98ac59a04f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:31.728159 8868 tablet_service.cc:1467] Processing CreateTablet for tablet 17defa6669d94e8bbcbcd2dd779b18f1 (DEFAULT_TABLE table=test-table [id=3239f7da4f0d4038a4ef3f563b1ead28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:31.728259 8720 tablet_service.cc:1467] Processing CreateTablet for tablet 17defa6669d94e8bbcbcd2dd779b18f1 (DEFAULT_TABLE table=test-table [id=3239f7da4f0d4038a4ef3f563b1ead28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:31.728317 9212 ts_tablet_manager.cc:1918] T 4b316a2376a94d32a93798165b0f402c P 166d007d7628419abf3b0f98ac59a04f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 11:55:31.728384 9212 log.cc:1196] T 4b316a2376a94d32a93798165b0f402c P 166d007d7628419abf3b0f98ac59a04f: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-0/wal/wals/4b316a2376a94d32a93798165b0f402c
I20260811 11:55:31.728449 8868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17defa6669d94e8bbcbcd2dd779b18f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:31.728493 8720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17defa6669d94e8bbcbcd2dd779b18f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:31.728626 9212 ts_tablet_manager.cc:1939] T 4b316a2376a94d32a93798165b0f402c P 166d007d7628419abf3b0f98ac59a04f: Deleting consensus metadata
I20260811 11:55:31.729282 8556 catalog_manager.cc:4841] TS 166d007d7628419abf3b0f98ac59a04f (127.6.143.129:45063): tablet 4b316a2376a94d32a93798165b0f402c (table test-table [id=a5cadfd8cb524f54aaa74b2150f36a38]) successfully deleted
I20260811 11:55:31.730858 9226 tablet_bootstrap.cc:492] T 17defa6669d94e8bbcbcd2dd779b18f1 P fed2e14718264870afb1a78b04a64ee7: Bootstrap starting.
I20260811 11:55:31.731230 9001 tablet_service.cc:1467] Processing CreateTablet for tablet 17defa6669d94e8bbcbcd2dd779b18f1 (DEFAULT_TABLE table=test-table [id=3239f7da4f0d4038a4ef3f563b1ead28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:31.731518 9001 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 17defa6669d94e8bbcbcd2dd779b18f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:31.731799 9226 tablet_bootstrap.cc:654] T 17defa6669d94e8bbcbcd2dd779b18f1 P fed2e14718264870afb1a78b04a64ee7: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:31.732052 9225 tablet_bootstrap.cc:492] T 17defa6669d94e8bbcbcd2dd779b18f1 P 166d007d7628419abf3b0f98ac59a04f: Bootstrap starting.
I20260811 11:55:31.732824 9225 tablet_bootstrap.cc:654] T 17defa6669d94e8bbcbcd2dd779b18f1 P 166d007d7628419abf3b0f98ac59a04f: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:31.733850 9224 tablet_bootstrap.cc:492] T 17defa6669d94e8bbcbcd2dd779b18f1 P a4d81298eb6e46bc95c4ab437d740735: Bootstrap starting.
I20260811 11:55:31.733932 9225 tablet_bootstrap.cc:492] T 17defa6669d94e8bbcbcd2dd779b18f1 P 166d007d7628419abf3b0f98ac59a04f: No bootstrap required, opened a new log
I20260811 11:55:31.734009 9225 ts_tablet_manager.cc:1397] T 17defa6669d94e8bbcbcd2dd779b18f1 P 166d007d7628419abf3b0f98ac59a04f: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 11:55:31.734373 9225 raft_consensus.cc:348] T 17defa6669d94e8bbcbcd2dd779b18f1 P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:31.734495 9225 raft_consensus.cc:374] T 17defa6669d94e8bbcbcd2dd779b18f1 P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:31.734531 9225 raft_consensus.cc:729] T 17defa6669d94e8bbcbcd2dd779b18f1 P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 166d007d7628419abf3b0f98ac59a04f, State: Initialized, Role: FOLLOWER
I20260811 11:55:31.734635 9225 consensus_queue.cc:260] T 17defa6669d94e8bbcbcd2dd779b18f1 P 166d007d7628419abf3b0f98ac59a04f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:31.734743 9224 tablet_bootstrap.cc:654] T 17defa6669d94e8bbcbcd2dd779b18f1 P a4d81298eb6e46bc95c4ab437d740735: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:31.734887 9225 ts_tablet_manager.cc:1428] T 17defa6669d94e8bbcbcd2dd779b18f1 P 166d007d7628419abf3b0f98ac59a04f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 11:55:31.735770 9226 tablet_bootstrap.cc:492] T 17defa6669d94e8bbcbcd2dd779b18f1 P fed2e14718264870afb1a78b04a64ee7: No bootstrap required, opened a new log
I20260811 11:55:31.735883 9226 ts_tablet_manager.cc:1397] T 17defa6669d94e8bbcbcd2dd779b18f1 P fed2e14718264870afb1a78b04a64ee7: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260811 11:55:31.736265 9224 tablet_bootstrap.cc:492] T 17defa6669d94e8bbcbcd2dd779b18f1 P a4d81298eb6e46bc95c4ab437d740735: No bootstrap required, opened a new log
I20260811 11:55:31.736369 9224 ts_tablet_manager.cc:1397] T 17defa6669d94e8bbcbcd2dd779b18f1 P a4d81298eb6e46bc95c4ab437d740735: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 11:55:31.736338 9226 raft_consensus.cc:348] T 17defa6669d94e8bbcbcd2dd779b18f1 P fed2e14718264870afb1a78b04a64ee7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:31.736485 9226 raft_consensus.cc:374] T 17defa6669d94e8bbcbcd2dd779b18f1 P fed2e14718264870afb1a78b04a64ee7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:31.736531 9226 raft_consensus.cc:729] T 17defa6669d94e8bbcbcd2dd779b18f1 P fed2e14718264870afb1a78b04a64ee7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fed2e14718264870afb1a78b04a64ee7, State: Initialized, Role: FOLLOWER
I20260811 11:55:31.736670 9226 consensus_queue.cc:260] T 17defa6669d94e8bbcbcd2dd779b18f1 P fed2e14718264870afb1a78b04a64ee7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:31.736810 9224 raft_consensus.cc:348] T 17defa6669d94e8bbcbcd2dd779b18f1 P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:31.736918 9224 raft_consensus.cc:374] T 17defa6669d94e8bbcbcd2dd779b18f1 P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:31.736955 9224 raft_consensus.cc:729] T 17defa6669d94e8bbcbcd2dd779b18f1 P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4d81298eb6e46bc95c4ab437d740735, State: Initialized, Role: FOLLOWER
I20260811 11:55:31.736996 9226 ts_tablet_manager.cc:1428] T 17defa6669d94e8bbcbcd2dd779b18f1 P fed2e14718264870afb1a78b04a64ee7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 11:55:31.737083 9224 consensus_queue.cc:260] T 17defa6669d94e8bbcbcd2dd779b18f1 P a4d81298eb6e46bc95c4ab437d740735 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:31.737368 9224 ts_tablet_manager.cc:1428] T 17defa6669d94e8bbcbcd2dd779b18f1 P a4d81298eb6e46bc95c4ab437d740735: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 11:55:31.970347 9202 raft_consensus.cc:482] T 17defa6669d94e8bbcbcd2dd779b18f1 P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:55:31.970508 9202 raft_consensus.cc:504] T 17defa6669d94e8bbcbcd2dd779b18f1 P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:31.970986 9202 leader_election.cc:288] T 17defa6669d94e8bbcbcd2dd779b18f1 P 166d007d7628419abf3b0f98ac59a04f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fed2e14718264870afb1a78b04a64ee7 (127.6.143.130:36051), a4d81298eb6e46bc95c4ab437d740735 (127.6.143.131:43925)
I20260811 11:55:31.971426 8888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17defa6669d94e8bbcbcd2dd779b18f1" candidate_uuid: "166d007d7628419abf3b0f98ac59a04f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fed2e14718264870afb1a78b04a64ee7" is_pre_election: true
I20260811 11:55:31.971467 9043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17defa6669d94e8bbcbcd2dd779b18f1" candidate_uuid: "166d007d7628419abf3b0f98ac59a04f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4d81298eb6e46bc95c4ab437d740735" is_pre_election: true
I20260811 11:55:31.971601 8888 raft_consensus.cc:2454] T 17defa6669d94e8bbcbcd2dd779b18f1 P fed2e14718264870afb1a78b04a64ee7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 166d007d7628419abf3b0f98ac59a04f in term 0.
I20260811 11:55:31.971601 9043 raft_consensus.cc:2454] T 17defa6669d94e8bbcbcd2dd779b18f1 P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 166d007d7628419abf3b0f98ac59a04f in term 0.
I20260811 11:55:31.971961 8672 leader_election.cc:302] T 17defa6669d94e8bbcbcd2dd779b18f1 P 166d007d7628419abf3b0f98ac59a04f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 166d007d7628419abf3b0f98ac59a04f, a4d81298eb6e46bc95c4ab437d740735, fed2e14718264870afb1a78b04a64ee7; no voters:
I20260811 11:55:31.972157 9202 raft_consensus.cc:2781] T 17defa6669d94e8bbcbcd2dd779b18f1 P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:55:31.972225 9202 raft_consensus.cc:482] T 17defa6669d94e8bbcbcd2dd779b18f1 P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:55:31.972254 9202 raft_consensus.cc:3037] T 17defa6669d94e8bbcbcd2dd779b18f1 P 166d007d7628419abf3b0f98ac59a04f [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:31.973308 9202 raft_consensus.cc:504] T 17defa6669d94e8bbcbcd2dd779b18f1 P 166d007d7628419abf3b0f98ac59a04f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:31.973636 9202 leader_election.cc:288] T 17defa6669d94e8bbcbcd2dd779b18f1 P 166d007d7628419abf3b0f98ac59a04f [CANDIDATE]: Term 1 election: Requested vote from peers fed2e14718264870afb1a78b04a64ee7 (127.6.143.130:36051), a4d81298eb6e46bc95c4ab437d740735 (127.6.143.131:43925)
I20260811 11:55:31.974025 8888 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17defa6669d94e8bbcbcd2dd779b18f1" candidate_uuid: "166d007d7628419abf3b0f98ac59a04f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fed2e14718264870afb1a78b04a64ee7"
I20260811 11:55:31.974013 9043 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "17defa6669d94e8bbcbcd2dd779b18f1" candidate_uuid: "166d007d7628419abf3b0f98ac59a04f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a4d81298eb6e46bc95c4ab437d740735"
I20260811 11:55:31.974141 8888 raft_consensus.cc:3037] T 17defa6669d94e8bbcbcd2dd779b18f1 P fed2e14718264870afb1a78b04a64ee7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:31.974141 9043 raft_consensus.cc:3037] T 17defa6669d94e8bbcbcd2dd779b18f1 P a4d81298eb6e46bc95c4ab437d740735 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:31.975282 8888 raft_consensus.cc:2454] T 17defa6669d94e8bbcbcd2dd779b18f1 P fed2e14718264870afb1a78b04a64ee7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 166d007d7628419abf3b0f98ac59a04f in term 1.
I20260811 11:55:31.975282 9043 raft_consensus.cc:2454] T 17defa6669d94e8bbcbcd2dd779b18f1 P a4d81298eb6e46bc95c4ab437d740735 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 166d007d7628419abf3b0f98ac59a04f in term 1.
I20260811 11:55:31.975581 8672 leader_election.cc:302] T 17defa6669d94e8bbcbcd2dd779b18f1 P 166d007d7628419abf3b0f98ac59a04f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 166d007d7628419abf3b0f98ac59a04f, a4d81298eb6e46bc95c4ab437d740735; no voters:
I20260811 11:55:31.975749 9202 raft_consensus.cc:2781] T 17defa6669d94e8bbcbcd2dd779b18f1 P 166d007d7628419abf3b0f98ac59a04f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:31.975838 9202 raft_consensus.cc:686] T 17defa6669d94e8bbcbcd2dd779b18f1 P 166d007d7628419abf3b0f98ac59a04f [term 1 LEADER]: Becoming Leader. State: Replica: 166d007d7628419abf3b0f98ac59a04f, State: Running, Role: LEADER
I20260811 11:55:31.976038 9202 consensus_queue.cc:237] T 17defa6669d94e8bbcbcd2dd779b18f1 P 166d007d7628419abf3b0f98ac59a04f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } }
I20260811 11:55:31.977809 8571 catalog_manager.cc:5495] T 17defa6669d94e8bbcbcd2dd779b18f1 P 166d007d7628419abf3b0f98ac59a04f reported cstate change: term changed from 0 to 1, leader changed from <none> to 166d007d7628419abf3b0f98ac59a04f (127.6.143.129). New cstate: current_term: 1 leader_uuid: "166d007d7628419abf3b0f98ac59a04f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "166d007d7628419abf3b0f98ac59a04f" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45063 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 } health_report { overall_health: UNKNOWN } } }
I20260811 11:55:32.038288 9244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:32.035245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33120 (local address 127.6.143.129:45063)
0811 11:55:32.035512 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:32.035519 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:32.035530 (+ 11us) server_negotiation.cc:407] Connection header received
0811 11:55:32.035568 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:32.035570 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:32.035608 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:32.035683 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:55:32.035958 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:32.036542 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:32.037310 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:32.037525 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:32.037611 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:32.037831 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:55:32.037900 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:55:32.037955 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:32.038058 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 11:55:32.038102 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":117,"threads_started":1}
I20260811 11:55:32.044292 8888 raft_consensus.cc:1261] T 17defa6669d94e8bbcbcd2dd779b18f1 P fed2e14718264870afb1a78b04a64ee7 [term 1 FOLLOWER]: Refusing update from remote peer 166d007d7628419abf3b0f98ac59a04f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:32.044312 9043 raft_consensus.cc:1261] T 17defa6669d94e8bbcbcd2dd779b18f1 P a4d81298eb6e46bc95c4ab437d740735 [term 1 FOLLOWER]: Refusing update from remote peer 166d007d7628419abf3b0f98ac59a04f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:32.044742 9202 consensus_queue.cc:1035] T 17defa6669d94e8bbcbcd2dd779b18f1 P 166d007d7628419abf3b0f98ac59a04f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a4d81298eb6e46bc95c4ab437d740735" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:32.044926 9184 consensus_queue.cc:1035] T 17defa6669d94e8bbcbcd2dd779b18f1 P 166d007d7628419abf3b0f98ac59a04f [LEADER]: Connected to new peer: Peer: permanent_uuid: "fed2e14718264870afb1a78b04a64ee7" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 11:55:32.058344 9190 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 11:55:33.606899 8572 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:55:33.607045 8572 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56586:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:55:33.609352 8572 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 016ab2aae5fd45419161a6186a9719b6: Sending DeleteTablet for 3 replicas of tablet 17defa6669d94e8bbcbcd2dd779b18f1
I20260811 11:55:33.609825 8720 tablet_service.cc:1514] Processing DeleteTablet for tablet 17defa6669d94e8bbcbcd2dd779b18f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:33 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:41936
I20260811 11:55:33.609949 8868 tablet_service.cc:1514] Processing DeleteTablet for tablet 17defa6669d94e8bbcbcd2dd779b18f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:33 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:50544
I20260811 11:55:33.609980 9001 tablet_service.cc:1514] Processing DeleteTablet for tablet 17defa6669d94e8bbcbcd2dd779b18f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:33 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:38508
I20260811 11:55:33.610224 9271 tablet_replica.cc:330] T 17defa6669d94e8bbcbcd2dd779b18f1 P 166d007d7628419abf3b0f98ac59a04f: stopping tablet replica
I20260811 11:55:33.610234 9272 tablet_replica.cc:330] T 17defa6669d94e8bbcbcd2dd779b18f1 P fed2e14718264870afb1a78b04a64ee7: stopping tablet replica
I20260811 11:55:33.610354 9271 raft_consensus.cc:2229] T 17defa6669d94e8bbcbcd2dd779b18f1 P 166d007d7628419abf3b0f98ac59a04f [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:55:33.610435 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 8652
I20260811 11:55:33.610354 9272 raft_consensus.cc:2229] T 17defa6669d94e8bbcbcd2dd779b18f1 P fed2e14718264870afb1a78b04a64ee7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:55:33.615057 9272 raft_consensus.cc:2258] T 17defa6669d94e8bbcbcd2dd779b18f1 P fed2e14718264870afb1a78b04a64ee7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:33.615326 9273 tablet_replica.cc:330] T 17defa6669d94e8bbcbcd2dd779b18f1 P a4d81298eb6e46bc95c4ab437d740735: stopping tablet replica
I20260811 11:55:33.615532 9273 raft_consensus.cc:2229] T 17defa6669d94e8bbcbcd2dd779b18f1 P a4d81298eb6e46bc95c4ab437d740735 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:55:33.615655 9273 raft_consensus.cc:2258] T 17defa6669d94e8bbcbcd2dd779b18f1 P a4d81298eb6e46bc95c4ab437d740735 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:33.615772 9272 ts_tablet_manager.cc:1905] T 17defa6669d94e8bbcbcd2dd779b18f1 P fed2e14718264870afb1a78b04a64ee7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:33.616326 9273 ts_tablet_manager.cc:1905] T 17defa6669d94e8bbcbcd2dd779b18f1 P a4d81298eb6e46bc95c4ab437d740735: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:33.618669 9272 ts_tablet_manager.cc:1918] T 17defa6669d94e8bbcbcd2dd779b18f1 P fed2e14718264870afb1a78b04a64ee7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 11:55:33.618794 9272 log.cc:1196] T 17defa6669d94e8bbcbcd2dd779b18f1 P fed2e14718264870afb1a78b04a64ee7: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-1/wal/wals/17defa6669d94e8bbcbcd2dd779b18f1
I20260811 11:55:33.619107 9272 ts_tablet_manager.cc:1939] T 17defa6669d94e8bbcbcd2dd779b18f1 P fed2e14718264870afb1a78b04a64ee7: Deleting consensus metadata
I20260811 11:55:33.619149 9273 ts_tablet_manager.cc:1918] T 17defa6669d94e8bbcbcd2dd779b18f1 P a4d81298eb6e46bc95c4ab437d740735: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 11:55:33.619217 9273 log.cc:1196] T 17defa6669d94e8bbcbcd2dd779b18f1 P a4d81298eb6e46bc95c4ab437d740735: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786449319379472-6718-0/minicluster-data/ts-2/wal/wals/17defa6669d94e8bbcbcd2dd779b18f1
I20260811 11:55:33.619484 9273 ts_tablet_manager.cc:1939] T 17defa6669d94e8bbcbcd2dd779b18f1 P a4d81298eb6e46bc95c4ab437d740735: Deleting consensus metadata
I20260811 11:55:33.619933 8557 catalog_manager.cc:4841] TS fed2e14718264870afb1a78b04a64ee7 (127.6.143.130:36051): tablet 17defa6669d94e8bbcbcd2dd779b18f1 (table test-table [id=3239f7da4f0d4038a4ef3f563b1ead28]) successfully deleted
I20260811 11:55:33.620195 8558 catalog_manager.cc:4841] TS a4d81298eb6e46bc95c4ab437d740735 (127.6.143.131:43925): tablet 17defa6669d94e8bbcbcd2dd779b18f1 (table test-table [id=3239f7da4f0d4038a4ef3f563b1ead28]) successfully deleted
W20260811 11:55:33.622946 8556 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.143.129:45063 (error 108)
I20260811 11:55:33.623208 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 8800
I20260811 11:55:33.625669 9275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:33.625269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:45063 (local address 127.0.0.1:33130)
0811 11:55:33.625518 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:33.625586 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.129:45063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":95,"threads_started":1}
W20260811 11:55:33.625809 8556 catalog_manager.cc:4584] TS 166d007d7628419abf3b0f98ac59a04f (127.6.143.129:45063): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 17defa6669d94e8bbcbcd2dd779b18f1: Network error: Client connection negotiation failed: client connection to 127.6.143.129:45063: connect: Connection refused (error 111)
I20260811 11:55:33.633030 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 8951
I20260811 11:55:33.642318 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 8539
2026-08-11T11:55:33Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6332 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:55:33 dist-test-slave-zkpd krb5kdc[9278](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:55:33 dist-test-slave-zkpd krb5kdc[9278](info): set up 2 sockets
Aug 11 11:55:33 dist-test-slave-zkpd krb5kdc[9278](info): commencing operation
krb5kdc: starting...
W20260811 11:55:34.733505 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:55:34 dist-test-slave-zkpd krb5kdc[9278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:55:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:55:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
I20260811 11:55:34.854308 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:43927
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:34737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:43927
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:34.967825 9291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:34.968092 9291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:34.968149 9291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:34.972170 9291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:55:34.972219 9291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:34.972239 9291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:34.972254 9291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:55:34.972309 9291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:55:34.977604 9291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:34737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:43927
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:43927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:34.978576 9291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:34.979597 9291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:34.988101 9299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:34.988205 9291 server_base.cc:1034] running on GCE node
W20260811 11:55:34.988126 9297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:34.988125 9301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:34.988991 9291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:34.990016 9291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:34.991190 9291 hybrid_clock.cc:648] HybridClock initialized: now 1786449334991162 us; error 40 us; skew 500 ppm
Aug 11 11:55:34 dist-test-slave-zkpd krb5kdc[9278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449334, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:34.998231 9291 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:55:34.999109 9291 webserver.cc:460] Webserver started at http://127.6.143.190:41967/ using document root <none> and password file <none>
I20260811 11:55:34.999720 9291 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:34.999805 9291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:35.000061 9291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:35.002068 9291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "30a97488167c4610bc2ff8b5a4c41f87"
format_stamp: "Formatted at 2026-08-11 11:55:34 on dist-test-slave-zkpd"
server_key: "601cf7a131bece73b66289a1669246af"
server_key_iv: "d4aedf631648c1ca9b7cb86128c3ed90"
server_key_version: "encryptionkey@0"
I20260811 11:55:35.002641 9291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "30a97488167c4610bc2ff8b5a4c41f87"
format_stamp: "Formatted at 2026-08-11 11:55:34 on dist-test-slave-zkpd"
server_key: "601cf7a131bece73b66289a1669246af"
server_key_iv: "d4aedf631648c1ca9b7cb86128c3ed90"
server_key_version: "encryptionkey@0"
I20260811 11:55:35.006435 9291 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 11:55:35.008814 9309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:35.009981 9291 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:55:35.010116 9291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "30a97488167c4610bc2ff8b5a4c41f87"
format_stamp: "Formatted at 2026-08-11 11:55:34 on dist-test-slave-zkpd"
server_key: "601cf7a131bece73b66289a1669246af"
server_key_iv: "d4aedf631648c1ca9b7cb86128c3ed90"
server_key_version: "encryptionkey@0"
I20260811 11:55:35.010221 9291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:35.037073 9291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:35.040086 9291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:35.040298 9291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:35.047531 9291 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:43927
I20260811 11:55:35.047559 9365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:43927 every 8 connection(s)
I20260811 11:55:35.048552 9291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:55:35.050129 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 9291
I20260811 11:55:35.050227 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:55:35.050532 6718 external_mini_cluster.cc:1428] Setting key 4a36dd8b1b94e4599c48a38b4cb86c85
I20260811 11:55:35.051611 9367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:35.058362 9367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30a97488167c4610bc2ff8b5a4c41f87: Bootstrap starting.
Aug 11 11:55:35 dist-test-slave-zkpd krb5kdc[9278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:35.060905 9367 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 30a97488167c4610bc2ff8b5a4c41f87: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:35.061616 9367 log.cc:824] T 00000000000000000000000000000000 P 30a97488167c4610bc2ff8b5a4c41f87: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:35.063737 9367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 30a97488167c4610bc2ff8b5a4c41f87: No bootstrap required, opened a new log
I20260811 11:55:35.065526 9370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:35.051952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57824 (local address 127.6.143.190:43927)
0811 11:55:35.052384 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:35.052404 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:35.052446 (+ 42us) server_negotiation.cc:407] Connection header received
0811 11:55:35.053298 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:35.053322 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:35.053742 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:35.054110 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:35.054756 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:35.056264 (+ 1508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:35.057080 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:35.057783 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:35.057952 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:35.060619 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:35.060624 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:35.060637 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:35.060662 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:35.062616 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:35.063089 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:35.063093 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:35.063097 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:35.063212 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:35.063497 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:35.063499 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:35.063500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:35.063878 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:35.064050 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:35.064320 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 11:55:35.064624 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":119,"threads_started":1}
I20260811 11:55:35.067216 9367 raft_consensus.cc:348] T 00000000000000000000000000000000 P 30a97488167c4610bc2ff8b5a4c41f87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30a97488167c4610bc2ff8b5a4c41f87" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 43927 } }
I20260811 11:55:35.067456 9367 raft_consensus.cc:374] T 00000000000000000000000000000000 P 30a97488167c4610bc2ff8b5a4c41f87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:35.067533 9367 raft_consensus.cc:729] T 00000000000000000000000000000000 P 30a97488167c4610bc2ff8b5a4c41f87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30a97488167c4610bc2ff8b5a4c41f87, State: Initialized, Role: FOLLOWER
I20260811 11:55:35.068017 9367 consensus_queue.cc:260] T 00000000000000000000000000000000 P 30a97488167c4610bc2ff8b5a4c41f87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30a97488167c4610bc2ff8b5a4c41f87" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 43927 } }
I20260811 11:55:35.068167 9367 raft_consensus.cc:388] T 00000000000000000000000000000000 P 30a97488167c4610bc2ff8b5a4c41f87 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:55:35.068233 9367 raft_consensus.cc:482] T 00000000000000000000000000000000 P 30a97488167c4610bc2ff8b5a4c41f87 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:55:35.068328 9367 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 30a97488167c4610bc2ff8b5a4c41f87 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:35.069152 9367 raft_consensus.cc:504] T 00000000000000000000000000000000 P 30a97488167c4610bc2ff8b5a4c41f87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30a97488167c4610bc2ff8b5a4c41f87" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 43927 } }
I20260811 11:55:35.069487 9367 leader_election.cc:302] T 00000000000000000000000000000000 P 30a97488167c4610bc2ff8b5a4c41f87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 30a97488167c4610bc2ff8b5a4c41f87; no voters:
I20260811 11:55:35.069775 9367 leader_election.cc:288] T 00000000000000000000000000000000 P 30a97488167c4610bc2ff8b5a4c41f87 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:55:35.069890 9372 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 30a97488167c4610bc2ff8b5a4c41f87 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:35.070113 9372 raft_consensus.cc:686] T 00000000000000000000000000000000 P 30a97488167c4610bc2ff8b5a4c41f87 [term 1 LEADER]: Becoming Leader. State: Replica: 30a97488167c4610bc2ff8b5a4c41f87, State: Running, Role: LEADER
I20260811 11:55:35.070482 9372 consensus_queue.cc:237] T 00000000000000000000000000000000 P 30a97488167c4610bc2ff8b5a4c41f87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30a97488167c4610bc2ff8b5a4c41f87" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 43927 } }
I20260811 11:55:35.070962 9367 sys_catalog.cc:564] T 00000000000000000000000000000000 P 30a97488167c4610bc2ff8b5a4c41f87 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:55:35.072114 9374 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30a97488167c4610bc2ff8b5a4c41f87 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 30a97488167c4610bc2ff8b5a4c41f87. Latest consensus state: current_term: 1 leader_uuid: "30a97488167c4610bc2ff8b5a4c41f87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30a97488167c4610bc2ff8b5a4c41f87" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 43927 } } }
I20260811 11:55:35.072244 9374 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30a97488167c4610bc2ff8b5a4c41f87 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:35.072551 9373 sys_catalog.cc:455] T 00000000000000000000000000000000 P 30a97488167c4610bc2ff8b5a4c41f87 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "30a97488167c4610bc2ff8b5a4c41f87" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "30a97488167c4610bc2ff8b5a4c41f87" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 43927 } } }
I20260811 11:55:35.072629 9383 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:55:35.072651 9373 sys_catalog.cc:458] T 00000000000000000000000000000000 P 30a97488167c4610bc2ff8b5a4c41f87 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:35.075606 9383 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:55:35.080684 9383 catalog_manager.cc:1347] Generated new cluster ID: ae579c70162242259ab2782f2d364708
I20260811 11:55:35.080745 9383 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:55:35.117360 9383 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:55:35.118588 9383 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:55:35.128751 9383 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 30a97488167c4610bc2ff8b5a4c41f87: Generated new TSK 0
I20260811 11:55:35.129552 9383 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
I20260811 11:55:35.158212 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:0
--local_ip_for_outbound_sockets=127.6.143.129
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:43927
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:34737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:35.270085 9395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:35.270316 9395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:35.270356 9395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:35.274621 9395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:35.274677 9395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:35.274732 9395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:55:35.280323 9395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:34737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:43927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:35.281247 9395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:35.282209 9395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:35.291520 9405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:35.291534 9403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:35.291625 9395 server_base.cc:1034] running on GCE node
W20260811 11:55:35.292423 9402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:35.292838 9395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:35.293457 9395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:35.294634 9395 hybrid_clock.cc:648] HybridClock initialized: now 1786449335294605 us; error 46 us; skew 500 ppm
Aug 11 11:55:35 dist-test-slave-zkpd krb5kdc[9278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449335, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:35.297461 9395 init.cc:377] Logged in from keytab as kudu/127.6.143.129@KRBTEST.COM (short username kudu)
I20260811 11:55:35.298282 9395 webserver.cc:460] Webserver started at http://127.6.143.129:33697/ using document root <none> and password file <none>
I20260811 11:55:35.298990 9395 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:35.299084 9395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:35.299288 9395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:35.301282 9395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/data/instance:
uuid: "8742a524a4914d21abd2a2ba1695c83a"
format_stamp: "Formatted at 2026-08-11 11:55:35 on dist-test-slave-zkpd"
server_key: "1d0b95889bd7a3a7917c92be59db81e2"
server_key_iv: "dcbf739df0ef78a73c5cfcdc1287c816"
server_key_version: "encryptionkey@0"
I20260811 11:55:35.301795 9395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance:
uuid: "8742a524a4914d21abd2a2ba1695c83a"
format_stamp: "Formatted at 2026-08-11 11:55:35 on dist-test-slave-zkpd"
server_key: "1d0b95889bd7a3a7917c92be59db81e2"
server_key_iv: "dcbf739df0ef78a73c5cfcdc1287c816"
server_key_version: "encryptionkey@0"
I20260811 11:55:35.305400 9395 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 11:55:35.307746 9412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:35.308974 9395 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:55:35.309113 9395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "8742a524a4914d21abd2a2ba1695c83a"
format_stamp: "Formatted at 2026-08-11 11:55:35 on dist-test-slave-zkpd"
server_key: "1d0b95889bd7a3a7917c92be59db81e2"
server_key_iv: "dcbf739df0ef78a73c5cfcdc1287c816"
server_key_version: "encryptionkey@0"
I20260811 11:55:35.309222 9395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:35.328872 9395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:35.332273 9395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:35.332482 9395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:35.333086 9395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:55:35.333937 9395 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:35.334008 9395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:35.334079 9395 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:35.334120 9395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:35.342944 9395 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:46707
I20260811 11:55:35.342993 9536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:46707 every 8 connection(s)
I20260811 11:55:35.343832 9395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:55:35.344787 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 9395
I20260811 11:55:35.344909 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance
I20260811 11:55:35.345194 6718 external_mini_cluster.cc:1428] Setting key 3721bfa2b1fd898dbb56b89473f1abc8
Aug 11 11:55:35 dist-test-slave-zkpd krb5kdc[9278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449335, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.143.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.130@KRBTEST.COM" created.
I20260811 11:55:35.359880 9370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:35.345864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:40865 (local address 127.6.143.190:43927)
0811 11:55:35.346007 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:35.346011 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:35.347079 (+ 1068us) server_negotiation.cc:407] Connection header received
0811 11:55:35.348505 (+ 1426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:35.348511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:35.348578 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:35.348675 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:35.350385 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:35.350918 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:35.352813 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:35.353047 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:35.353129 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:35.356045 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:35.356048 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:35.356050 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:35.356074 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:35.357567 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:35.358094 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:35.358097 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:35.358098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:35.358203 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:35.358569 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:35.358575 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:35.358577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:35.358769 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:35.358873 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:35.359472 (+ 599us) server_negotiation.cc:299] Negotiation successful
0811 11:55:35.359627 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 11:55:35.360541 9539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:35.346281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:43927 (local address 127.6.143.129:40865)
0811 11:55:35.346923 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:35.346978 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:35.348289 (+ 1311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:35.348931 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:35.348943 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:35.349480 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:35.350210 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:35.350233 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:35.351054 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:35.351061 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:35.352683 (+ 1622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:35.352699 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:35.353212 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:35.353219 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:35.353484 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:35.354155 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:35.354178 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:35.355821 (+ 1643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:35.357688 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:35.357694 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:35.357709 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:35.357978 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:35.358312 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:35.358314 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:35.358316 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:35.358463 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:35.358898 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:35.358905 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:35.359202 (+ 297us) client_negotiation.cc:769] Sending connection context
0811 11:55:35.359356 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 11:55:35.359614 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":146,"threads_started":1}
I20260811 11:55:35.361886 9537 heartbeater.cc:347] Connected to a master server at 127.6.143.190:43927
I20260811 11:55:35.362141 9537 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:35.362633 9537 heartbeater.cc:511] Master 127.6.143.190:43927 requested a full tablet report, sending...
I20260811 11:55:35.364456 9328 ts_manager.cc:194] Registered new tserver with Master: 8742a524a4914d21abd2a2ba1695c83a (127.6.143.129:46707)
I20260811 11:55:35.366142 9328 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.129@KRBTEST.COM'} at 127.6.143.129:40865
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
I20260811 11:55:35.374893 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:0
--local_ip_for_outbound_sockets=127.6.143.130
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:43927
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:34737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:35.487206 9542 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:35.487452 9542 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:35.487491 9542 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:35.491328 9542 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:35.491379 9542 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:35.491433 9542 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:55:35.498263 9542 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:34737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:43927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9542
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:35.499233 9542 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:35.500252 9542 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:35.509290 9547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:35.509289 9548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:35.509290 9550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:35.509500 9542 server_base.cc:1034] running on GCE node
I20260811 11:55:35.510052 9542 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:35.510605 9542 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:35.511775 9542 hybrid_clock.cc:648] HybridClock initialized: now 1786449335511741 us; error 50 us; skew 500 ppm
Aug 11 11:55:35 dist-test-slave-zkpd krb5kdc[9278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449335, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:35.514448 9542 init.cc:377] Logged in from keytab as kudu/127.6.143.130@KRBTEST.COM (short username kudu)
I20260811 11:55:35.515286 9542 webserver.cc:460] Webserver started at http://127.6.143.130:42437/ using document root <none> and password file <none>
I20260811 11:55:35.515825 9542 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:35.515934 9542 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:35.516162 9542 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:35.518077 9542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/data/instance:
uuid: "ac5937f10b5a4fa48b0fc8175bdb0592"
format_stamp: "Formatted at 2026-08-11 11:55:35 on dist-test-slave-zkpd"
server_key: "d75388372d13500a5983972479554d11"
server_key_iv: "a64c3586533a27d57afcda911e6369e3"
server_key_version: "encryptionkey@0"
I20260811 11:55:35.518623 9542 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance:
uuid: "ac5937f10b5a4fa48b0fc8175bdb0592"
format_stamp: "Formatted at 2026-08-11 11:55:35 on dist-test-slave-zkpd"
server_key: "d75388372d13500a5983972479554d11"
server_key_iv: "a64c3586533a27d57afcda911e6369e3"
server_key_version: "encryptionkey@0"
I20260811 11:55:35.522112 9542 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260811 11:55:35.524405 9558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:35.525581 9542 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:35.525673 9542 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "ac5937f10b5a4fa48b0fc8175bdb0592"
format_stamp: "Formatted at 2026-08-11 11:55:35 on dist-test-slave-zkpd"
server_key: "d75388372d13500a5983972479554d11"
server_key_iv: "a64c3586533a27d57afcda911e6369e3"
server_key_version: "encryptionkey@0"
I20260811 11:55:35.525748 9542 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:35.541674 9542 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:35.544359 9542 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:35.544508 9542 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:35.545022 9542 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:55:35.545792 9542 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:35.545837 9542 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:35.545874 9542 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:35.545895 9542 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:35.554581 9542 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:38749
I20260811 11:55:35.554615 9681 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:38749 every 8 connection(s)
I20260811 11:55:35.555429 9542 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
I20260811 11:55:35.561115 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 9542
I20260811 11:55:35.561231 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance
I20260811 11:55:35.561508 6718 external_mini_cluster.cc:1428] Setting key fd79a21d07397a2073a9bd0e537f673b
Aug 11 11:55:35 dist-test-slave-zkpd krb5kdc[9278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449335, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:35.569612 9370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:35.557141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:42615 (local address 127.6.143.190:43927)
0811 11:55:35.557270 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:35.557274 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:35.558010 (+ 736us) server_negotiation.cc:407] Connection header received
0811 11:55:35.559376 (+ 1366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:35.559379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:35.559458 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:35.559566 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:35.560774 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:35.561583 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:35.562918 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:35.563158 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:35.563266 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:35.565832 (+ 2566us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:35.565839 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:35.565843 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:35.565876 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:35.567368 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:35.568000 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:35.568003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:35.568004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:35.568078 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:35.568505 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:35.568507 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:35.568509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:35.568690 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:35.568766 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:35.569406 (+ 640us) server_negotiation.cc:299] Negotiation successful
0811 11:55:35.569500 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 11:55:35.570875 9685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:35.557439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:43927 (local address 127.6.143.130:42615)
0811 11:55:35.557865 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:35.557905 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:35.559156 (+ 1251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:35.559759 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:35.559766 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:35.560171 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:35.560616 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:35.560629 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:35.561708 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:35.561711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:35.562784 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:35.562795 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:35.563302 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:35.563307 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:35.563485 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:35.564072 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:35.564087 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:35.565660 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:35.567478 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:35.567487 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:35.567504 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:35.567872 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:35.568193 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:35.568199 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:35.568204 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:35.568410 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:35.568797 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:35.568804 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:35.569124 (+ 320us) client_negotiation.cc:769] Sending connection context
0811 11:55:35.569369 (+ 245us) client_negotiation.cc:240] Negotiation successful
0811 11:55:35.569735 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":109,"threads_started":1}
I20260811 11:55:35.572187 9682 heartbeater.cc:347] Connected to a master server at 127.6.143.190:43927
I20260811 11:55:35.572428 9682 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:35.572907 9682 heartbeater.cc:511] Master 127.6.143.190:43927 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.143.131@KRBTEST.COM; defaulting to no policy
I20260811 11:55:35.574095 9328 ts_manager.cc:194] Registered new tserver with Master: ac5937f10b5a4fa48b0fc8175bdb0592 (127.6.143.130:38749)
I20260811 11:55:35.574615 9328 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.130@KRBTEST.COM'} at 127.6.143.130:42615
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
I20260811 11:55:35.588825 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:0
--local_ip_for_outbound_sockets=127.6.143.131
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:43927
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:34737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:35.700322 9688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:35.700557 9688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:35.700598 9688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:35.704506 9688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:35.704553 9688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:35.704607 9688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:55:35.709590 9688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:34737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:43927
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:35.710455 9688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:35.711344 9688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:35.720679 9695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:35.720692 9698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:35.720692 9694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:35.720860 9688 server_base.cc:1034] running on GCE node
I20260811 11:55:35.721323 9688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:35.721918 9688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:35.723132 9688 hybrid_clock.cc:648] HybridClock initialized: now 1786449335723102 us; error 27 us; skew 500 ppm
Aug 11 11:55:35 dist-test-slave-zkpd krb5kdc[9278](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449335, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:35.725853 9688 init.cc:377] Logged in from keytab as kudu/127.6.143.131@KRBTEST.COM (short username kudu)
I20260811 11:55:35.726598 9688 webserver.cc:460] Webserver started at http://127.6.143.131:46125/ using document root <none> and password file <none>
I20260811 11:55:35.727120 9688 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:35.727180 9688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:35.727334 9688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:35.729398 9688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/data/instance:
uuid: "dc2f686409ae4408bb695ef204cc5877"
format_stamp: "Formatted at 2026-08-11 11:55:35 on dist-test-slave-zkpd"
server_key: "c8a7640309e16dc7495dc8991e4e16b8"
server_key_iv: "58afc71d1000398c8d8f7c099a255335"
server_key_version: "encryptionkey@0"
I20260811 11:55:35.729944 9688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance:
uuid: "dc2f686409ae4408bb695ef204cc5877"
format_stamp: "Formatted at 2026-08-11 11:55:35 on dist-test-slave-zkpd"
server_key: "c8a7640309e16dc7495dc8991e4e16b8"
server_key_iv: "58afc71d1000398c8d8f7c099a255335"
server_key_version: "encryptionkey@0"
I20260811 11:55:35.733419 9688 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 11:55:35.735627 9705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:35.736769 9688 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:35.736862 9688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "dc2f686409ae4408bb695ef204cc5877"
format_stamp: "Formatted at 2026-08-11 11:55:35 on dist-test-slave-zkpd"
server_key: "c8a7640309e16dc7495dc8991e4e16b8"
server_key_iv: "58afc71d1000398c8d8f7c099a255335"
server_key_version: "encryptionkey@0"
I20260811 11:55:35.736937 9688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:35.762081 9688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:35.765154 9688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:35.765297 9688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:35.765787 9688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:55:35.766522 9688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:35.766597 9688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:35.766666 9688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:35.766690 9688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:35.775961 9688 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:38819
I20260811 11:55:35.776047 9826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:38819 every 8 connection(s)
I20260811 11:55:35.776844 9688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:55:35.784987 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 9688
I20260811 11:55:35.785154 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance
I20260811 11:55:35.785494 6718 external_mini_cluster.cc:1428] Setting key e28d4e2923cb47ed6377e2b334643c92
Aug 11 11:55:35 dist-test-slave-zkpd krb5kdc[9278](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449335, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:35.791572 9370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:35.778567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:54829 (local address 127.6.143.190:43927)
0811 11:55:35.778706 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:35.778709 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:35.779524 (+ 815us) server_negotiation.cc:407] Connection header received
0811 11:55:35.780950 (+ 1426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:35.780954 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:35.781013 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:35.781116 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:35.782256 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:35.782784 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:35.784393 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:35.784786 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:35.784929 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:35.788309 (+ 3380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:35.788313 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:35.788316 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:35.788343 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:35.789548 (+ 1205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:35.790060 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:35.790064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:35.790065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:35.790152 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:35.790502 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:35.790506 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:35.790508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:35.790694 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:35.790796 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:35.791290 (+ 494us) server_negotiation.cc:299] Negotiation successful
0811 11:55:35.791413 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 11:55:35.792444 9831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:35.778941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:43927 (local address 127.6.143.131:54829)
0811 11:55:35.779366 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:35.779418 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:35.780735 (+ 1317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:35.781293 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:35.781300 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:35.781642 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:35.782112 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:35.782126 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:35.782908 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:35.782914 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:35.784198 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:35.784209 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:35.785347 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:35.785357 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:35.785572 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:35.786302 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:35.786322 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:35.788147 (+ 1825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:35.789666 (+ 1519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:35.789672 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:35.789687 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:35.789950 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:35.790244 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:35.790248 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:35.790250 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:35.790399 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:35.790810 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:35.790819 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:35.791052 (+ 233us) client_negotiation.cc:769] Sending connection context
0811 11:55:35.791233 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 11:55:35.791592 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":107,"threads_started":1}
I20260811 11:55:35.793493 9828 heartbeater.cc:347] Connected to a master server at 127.6.143.190:43927
I20260811 11:55:35.793747 9828 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:35.794229 9828 heartbeater.cc:511] Master 127.6.143.190:43927 requested a full tablet report, sending...
I20260811 11:55:35.795178 9328 ts_manager.cc:194] Registered new tserver with Master: dc2f686409ae4408bb695ef204cc5877 (127.6.143.131:38819)
I20260811 11:55:35.795701 9328 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.131@KRBTEST.COM'} at 127.6.143.131:54829
I20260811 11:55:35.799219 6718 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 11:55:35.813062 9370 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:55:35.811634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57834 (local address 127.6.143.190:43927)
0811 11:55:35.811775 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:35.811782 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:35.811979 (+ 197us) server_negotiation.cc:407] Connection header received
0811 11:55:35.812098 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:35.812100 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:35.812141 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:35.812208 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:35.812938 (+ 730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57834: BlockingRecv error: recv got EOF from 127.0.0.1:57834 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 11:55:35.814414 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 9395
I20260811 11:55:35.824002 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 9542
I20260811 11:55:35.832209 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 9688
I20260811 11:55:35.839502 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 9291
2026-08-11T11:55:35Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2196 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:55:35 dist-test-slave-zkpd krb5kdc[9842](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:55:35 dist-test-slave-zkpd krb5kdc[9842](info): set up 2 sockets
Aug 11 11:55:35 dist-test-slave-zkpd krb5kdc[9842](info): commencing operation
krb5kdc: starting...
W20260811 11:55:36.932960 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:55:37 dist-test-slave-zkpd krb5kdc[9842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:55:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:55:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
I20260811 11:55:37.056942 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:35501
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:45209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:35501
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:37.169006 9857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:37.169267 9857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:37.169324 9857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:37.173207 9857 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:55:37.173254 9857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:37.173272 9857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:37.173290 9857 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:55:37.173305 9857 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:55:37.178409 9857 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:45209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:35501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:35501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:37.179425 9857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:37.180464 9857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:37.188766 9866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:37.188771 9864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:37.188731 9862 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:37.189023 9857 server_base.cc:1034] running on GCE node
I20260811 11:55:37.189545 9857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:37.190618 9857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:37.191809 9857 hybrid_clock.cc:648] HybridClock initialized: now 1786449337191778 us; error 59 us; skew 500 ppm
Aug 11 11:55:37 dist-test-slave-zkpd krb5kdc[9842](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449337, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:37.194531 9857 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:55:37.195422 9857 webserver.cc:460] Webserver started at http://127.6.143.190:33705/ using document root <none> and password file <none>
I20260811 11:55:37.196031 9857 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:37.196123 9857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:37.196370 9857 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:37.198348 9857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "789594a951194ba8a4599bcd3bdca714"
format_stamp: "Formatted at 2026-08-11 11:55:37 on dist-test-slave-zkpd"
server_key: "c309dd883c7521c526c76f081c46c066"
server_key_iv: "337c6d95909a06a8fc68ea2f19ae61d0"
server_key_version: "encryptionkey@0"
I20260811 11:55:37.198901 9857 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "789594a951194ba8a4599bcd3bdca714"
format_stamp: "Formatted at 2026-08-11 11:55:37 on dist-test-slave-zkpd"
server_key: "c309dd883c7521c526c76f081c46c066"
server_key_iv: "337c6d95909a06a8fc68ea2f19ae61d0"
server_key_version: "encryptionkey@0"
I20260811 11:55:37.202452 9857 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260811 11:55:37.204864 9874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:37.206158 9857 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:55:37.206303 9857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "789594a951194ba8a4599bcd3bdca714"
format_stamp: "Formatted at 2026-08-11 11:55:37 on dist-test-slave-zkpd"
server_key: "c309dd883c7521c526c76f081c46c066"
server_key_iv: "337c6d95909a06a8fc68ea2f19ae61d0"
server_key_version: "encryptionkey@0"
I20260811 11:55:37.206408 9857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:37.221598 9857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:37.224319 9857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:37.224502 9857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:37.231444 9857 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:35501
I20260811 11:55:37.231729 9930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:35501 every 8 connection(s)
I20260811 11:55:37.232472 9857 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:55:37.232968 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 9857
I20260811 11:55:37.233124 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:55:37.233392 6718 external_mini_cluster.cc:1428] Setting key e923f7a2165f0bef0ced4522366cea4c
I20260811 11:55:37.235641 9931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:37.242556 9931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 789594a951194ba8a4599bcd3bdca714: Bootstrap starting.
Aug 11 11:55:37 dist-test-slave-zkpd krb5kdc[9842](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449337, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:37.245124 9931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 789594a951194ba8a4599bcd3bdca714: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:37.246058 9931 log.cc:824] T 00000000000000000000000000000000 P 789594a951194ba8a4599bcd3bdca714: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:37.247701 9935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:37.234738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54604 (local address 127.6.143.190:35501)
0811 11:55:37.235155 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:37.235170 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:37.235207 (+ 37us) server_negotiation.cc:407] Connection header received
0811 11:55:37.235927 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:37.235959 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:37.236293 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:37.236568 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:37.237520 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:37.238535 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:37.239751 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:37.240280 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:37.240398 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:37.243237 (+ 2839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:37.243242 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:37.243255 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:37.243282 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:37.244972 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:37.245479 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:37.245484 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:37.245488 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:37.245600 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:37.245881 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:37.245882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:37.245883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:37.246276 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:37.246410 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:37.246767 (+ 357us) server_negotiation.cc:299] Negotiation successful
0811 11:55:37.247034 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":143,"threads_started":1}
I20260811 11:55:37.248173 9931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 789594a951194ba8a4599bcd3bdca714: No bootstrap required, opened a new log
I20260811 11:55:37.250633 6718 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:55:37.252231 9931 raft_consensus.cc:348] T 00000000000000000000000000000000 P 789594a951194ba8a4599bcd3bdca714 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "789594a951194ba8a4599bcd3bdca714" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 35501 } }
I20260811 11:55:37.252461 9931 raft_consensus.cc:374] T 00000000000000000000000000000000 P 789594a951194ba8a4599bcd3bdca714 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:37.252538 9931 raft_consensus.cc:729] T 00000000000000000000000000000000 P 789594a951194ba8a4599bcd3bdca714 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 789594a951194ba8a4599bcd3bdca714, State: Initialized, Role: FOLLOWER
I20260811 11:55:37.253089 9931 consensus_queue.cc:260] T 00000000000000000000000000000000 P 789594a951194ba8a4599bcd3bdca714 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "789594a951194ba8a4599bcd3bdca714" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 35501 } }
I20260811 11:55:37.253264 9931 raft_consensus.cc:388] T 00000000000000000000000000000000 P 789594a951194ba8a4599bcd3bdca714 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:55:37.253342 9931 raft_consensus.cc:482] T 00000000000000000000000000000000 P 789594a951194ba8a4599bcd3bdca714 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:55:37.253453 9931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 789594a951194ba8a4599bcd3bdca714 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:37.254711 9931 raft_consensus.cc:504] T 00000000000000000000000000000000 P 789594a951194ba8a4599bcd3bdca714 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "789594a951194ba8a4599bcd3bdca714" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 35501 } }
I20260811 11:55:37.255048 9931 leader_election.cc:302] T 00000000000000000000000000000000 P 789594a951194ba8a4599bcd3bdca714 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 789594a951194ba8a4599bcd3bdca714; no voters:
I20260811 11:55:37.255375 9931 leader_election.cc:288] T 00000000000000000000000000000000 P 789594a951194ba8a4599bcd3bdca714 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:55:37.255442 9938 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 789594a951194ba8a4599bcd3bdca714 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:37.255676 9938 raft_consensus.cc:686] T 00000000000000000000000000000000 P 789594a951194ba8a4599bcd3bdca714 [term 1 LEADER]: Becoming Leader. State: Replica: 789594a951194ba8a4599bcd3bdca714, State: Running, Role: LEADER
I20260811 11:55:37.256052 9938 consensus_queue.cc:237] T 00000000000000000000000000000000 P 789594a951194ba8a4599bcd3bdca714 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "789594a951194ba8a4599bcd3bdca714" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 35501 } }
I20260811 11:55:37.256675 9931 sys_catalog.cc:564] T 00000000000000000000000000000000 P 789594a951194ba8a4599bcd3bdca714 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:55:37.257278 9940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 789594a951194ba8a4599bcd3bdca714 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 789594a951194ba8a4599bcd3bdca714. Latest consensus state: current_term: 1 leader_uuid: "789594a951194ba8a4599bcd3bdca714" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "789594a951194ba8a4599bcd3bdca714" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 35501 } } }
I20260811 11:55:37.257510 9940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 789594a951194ba8a4599bcd3bdca714 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:37.257974 9946 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:55:37.259066 9939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 789594a951194ba8a4599bcd3bdca714 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "789594a951194ba8a4599bcd3bdca714" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "789594a951194ba8a4599bcd3bdca714" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 35501 } } }
I20260811 11:55:37.259176 9939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 789594a951194ba8a4599bcd3bdca714 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:37.261830 9946 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:55:37.267961 9946 catalog_manager.cc:1347] Generated new cluster ID: df8dddbb2c6f4dea98fc65efe90c4572
I20260811 11:55:37.268026 9946 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:55:37.272456 9935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:37.266591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54612 (local address 127.6.143.190:35501)
0811 11:55:37.266749 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:37.266754 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:37.267058 (+ 304us) server_negotiation.cc:407] Connection header received
0811 11:55:37.267229 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:37.267232 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:37.267276 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:37.267357 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:37.268040 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:37.268826 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:37.270000 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:37.270422 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:37.270546 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:37.271434 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:37.271438 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:37.271441 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:55:37.272015 (+ 574us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:37.272050 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:55:37.272070 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:37.272163 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:37.272262 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 11:55:37.272317 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 11:55:37.285024 9946 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:55:37.286149 9946 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:55:37.292601 9946 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 789594a951194ba8a4599bcd3bdca714: Generated new TSK 0
I20260811 11:55:37.293318 9946 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:55:37.301487 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 9857
2026-08-11T11:55:37Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1459 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:55:37 dist-test-slave-zkpd krb5kdc[9966](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:55:37 dist-test-slave-zkpd krb5kdc[9966](info): set up 2 sockets
Aug 11 11:55:37 dist-test-slave-zkpd krb5kdc[9966](info): commencing operation
krb5kdc: starting...
W20260811 11:55:38.380967 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:55:38 dist-test-slave-zkpd krb5kdc[9966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:55:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:55:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
I20260811 11:55:38.508667 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:46147
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:45051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:46147
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:38.622484 9980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:38.622731 9980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:38.622773 9980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:38.626820 9980 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:55:38.626876 9980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:38.626895 9980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:38.626911 9980 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:55:38.626927 9980 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:55:38.632136 9980 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:45051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:46147
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:46147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:38.633158 9980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:38.634192 9980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:38.642587 9989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:38.642609 9980 server_base.cc:1034] running on GCE node
W20260811 11:55:38.642765 9986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:38.642921 9985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:38.643550 9980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:38.644691 9980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:38.645884 9980 hybrid_clock.cc:648] HybridClock initialized: now 1786449338645849 us; error 44 us; skew 500 ppm
Aug 11 11:55:38 dist-test-slave-zkpd krb5kdc[9966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449338, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:38.648583 9980 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:55:38.649516 9980 webserver.cc:460] Webserver started at http://127.6.143.190:42327/ using document root <none> and password file <none>
I20260811 11:55:38.650099 9980 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:38.650190 9980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:38.650425 9980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:38.652601 9980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "fdff759f421b478e95faab1bb5fa0617"
format_stamp: "Formatted at 2026-08-11 11:55:38 on dist-test-slave-zkpd"
server_key: "dce37cc363db2319ae6e91e4a608d58d"
server_key_iv: "23c7748c233c410347a217c8b5412645"
server_key_version: "encryptionkey@0"
I20260811 11:55:38.653193 9980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "fdff759f421b478e95faab1bb5fa0617"
format_stamp: "Formatted at 2026-08-11 11:55:38 on dist-test-slave-zkpd"
server_key: "dce37cc363db2319ae6e91e4a608d58d"
server_key_iv: "23c7748c233c410347a217c8b5412645"
server_key_version: "encryptionkey@0"
I20260811 11:55:38.657140 9980 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 11:55:38.659708 9996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:38.661032 9980 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 11:55:38.661185 9980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "fdff759f421b478e95faab1bb5fa0617"
format_stamp: "Formatted at 2026-08-11 11:55:38 on dist-test-slave-zkpd"
server_key: "dce37cc363db2319ae6e91e4a608d58d"
server_key_iv: "23c7748c233c410347a217c8b5412645"
server_key_version: "encryptionkey@0"
I20260811 11:55:38.661301 9980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:38.679570 9980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:38.682515 9980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:38.682711 9980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:38.689664 9980 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:46147
I20260811 11:55:38.689960 10051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:46147 every 8 connection(s)
I20260811 11:55:38.690651 9980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:55:38.693542 10052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:38.695282 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 9980
I20260811 11:55:38.695429 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:55:38.695704 6718 external_mini_cluster.cc:1428] Setting key f6c956e949f109338444bbce8c22ffa7
I20260811 11:55:38.699811 10052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdff759f421b478e95faab1bb5fa0617: Bootstrap starting.
I20260811 11:55:38.702037 10052 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdff759f421b478e95faab1bb5fa0617: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:38.702756 10052 log.cc:824] T 00000000000000000000000000000000 P fdff759f421b478e95faab1bb5fa0617: Log is configured to *not* fsync() on all Append() calls
Aug 11 11:55:38 dist-test-slave-zkpd krb5kdc[9966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449338, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:38.704931 10052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdff759f421b478e95faab1bb5fa0617: No bootstrap required, opened a new log
I20260811 11:55:38.708774 10052 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdff759f421b478e95faab1bb5fa0617 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdff759f421b478e95faab1bb5fa0617" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46147 } }
I20260811 11:55:38.709026 10052 raft_consensus.cc:374] T 00000000000000000000000000000000 P fdff759f421b478e95faab1bb5fa0617 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:38.709100 10052 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdff759f421b478e95faab1bb5fa0617 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdff759f421b478e95faab1bb5fa0617, State: Initialized, Role: FOLLOWER
I20260811 11:55:38.709579 10052 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdff759f421b478e95faab1bb5fa0617 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdff759f421b478e95faab1bb5fa0617" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46147 } }
I20260811 11:55:38.709703 10052 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdff759f421b478e95faab1bb5fa0617 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:55:38.709782 10052 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdff759f421b478e95faab1bb5fa0617 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:55:38.709882 10052 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdff759f421b478e95faab1bb5fa0617 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:38.709899 10055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:38.697169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59244 (local address 127.6.143.190:46147)
0811 11:55:38.697619 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:38.697640 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:38.697675 (+ 35us) server_negotiation.cc:407] Connection header received
0811 11:55:38.698310 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:38.698326 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:38.698646 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:38.698986 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:38.699814 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:38.700851 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:38.701728 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:38.702259 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:38.702423 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:38.705171 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:38.705176 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:38.705189 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:38.705215 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:38.706915 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:38.707388 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:38.707393 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:38.707397 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:38.707500 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:38.707786 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:38.707788 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:38.707789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:38.708208 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:38.708369 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:38.708686 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 11:55:38.709029 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":151,"threads_started":1}
I20260811 11:55:38.710739 10052 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdff759f421b478e95faab1bb5fa0617 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdff759f421b478e95faab1bb5fa0617" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46147 } }
I20260811 11:55:38.711118 10052 leader_election.cc:302] T 00000000000000000000000000000000 P fdff759f421b478e95faab1bb5fa0617 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdff759f421b478e95faab1bb5fa0617; no voters:
I20260811 11:55:38.711449 10052 leader_election.cc:288] T 00000000000000000000000000000000 P fdff759f421b478e95faab1bb5fa0617 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:55:38.711560 10057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdff759f421b478e95faab1bb5fa0617 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:38.711788 10057 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdff759f421b478e95faab1bb5fa0617 [term 1 LEADER]: Becoming Leader. State: Replica: fdff759f421b478e95faab1bb5fa0617, State: Running, Role: LEADER
I20260811 11:55:38.712178 10057 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdff759f421b478e95faab1bb5fa0617 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdff759f421b478e95faab1bb5fa0617" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46147 } }
I20260811 11:55:38.712656 10052 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdff759f421b478e95faab1bb5fa0617 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:55:38.713313 10057 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdff759f421b478e95faab1bb5fa0617 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdff759f421b478e95faab1bb5fa0617. Latest consensus state: current_term: 1 leader_uuid: "fdff759f421b478e95faab1bb5fa0617" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdff759f421b478e95faab1bb5fa0617" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46147 } } }
I20260811 11:55:38.713434 10057 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdff759f421b478e95faab1bb5fa0617 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:38.713894 10058 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdff759f421b478e95faab1bb5fa0617 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fdff759f421b478e95faab1bb5fa0617" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdff759f421b478e95faab1bb5fa0617" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46147 } } }
I20260811 11:55:38.714010 10058 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdff759f421b478e95faab1bb5fa0617 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:38.714365 10064 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:55:38.717846 10064 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:55:38.723026 10064 catalog_manager.cc:1347] Generated new cluster ID: b4bdfc257744485b8c6168ac339698a6
I20260811 11:55:38.723088 10064 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:55:38.732110 10064 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:55:38.732971 10064 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:55:38.736631 10064 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fdff759f421b478e95faab1bb5fa0617: Generated new TSK 0
I20260811 11:55:38.737219 10064 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
I20260811 11:55:38.771822 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:0
--local_ip_for_outbound_sockets=127.6.143.129
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:46147
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:45051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:38.886219 10081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:38.886458 10081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:38.886521 10081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:38.890491 10081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:38.890551 10081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:38.890606 10081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:55:38.895735 10081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:45051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:46147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:38.896728 10081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:38.897662 10081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:38.907135 10087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:38.907176 10086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:38.907323 10089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:38.907580 10081 server_base.cc:1034] running on GCE node
I20260811 11:55:38.908020 10081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:38.908640 10081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:38.909819 10081 hybrid_clock.cc:648] HybridClock initialized: now 1786449338909802 us; error 28 us; skew 500 ppm
Aug 11 11:55:38 dist-test-slave-zkpd krb5kdc[9966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449338, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:38.912456 10081 init.cc:377] Logged in from keytab as kudu/127.6.143.129@KRBTEST.COM (short username kudu)
I20260811 11:55:38.913300 10081 webserver.cc:460] Webserver started at http://127.6.143.129:38155/ using document root <none> and password file <none>
I20260811 11:55:38.913975 10081 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:38.914047 10081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:38.914273 10081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:38.916416 10081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/data/instance:
uuid: "ffa65a449c83433f8ac3c21bbc873189"
format_stamp: "Formatted at 2026-08-11 11:55:38 on dist-test-slave-zkpd"
server_key: "218fef73fa8ebe236d9a6cf364b67d6b"
server_key_iv: "1fd2aff8720e6bb71dc2f5d3bbbf9953"
server_key_version: "encryptionkey@0"
I20260811 11:55:38.916963 10081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance:
uuid: "ffa65a449c83433f8ac3c21bbc873189"
format_stamp: "Formatted at 2026-08-11 11:55:38 on dist-test-slave-zkpd"
server_key: "218fef73fa8ebe236d9a6cf364b67d6b"
server_key_iv: "1fd2aff8720e6bb71dc2f5d3bbbf9953"
server_key_version: "encryptionkey@0"
I20260811 11:55:38.920631 10081 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 11:55:38.922973 10096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:38.924222 10081 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260811 11:55:38.924368 10081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "ffa65a449c83433f8ac3c21bbc873189"
format_stamp: "Formatted at 2026-08-11 11:55:38 on dist-test-slave-zkpd"
server_key: "218fef73fa8ebe236d9a6cf364b67d6b"
server_key_iv: "1fd2aff8720e6bb71dc2f5d3bbbf9953"
server_key_version: "encryptionkey@0"
I20260811 11:55:38.924458 10081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:38.937240 10081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:38.939930 10081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:38.940116 10081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:38.940678 10081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:55:38.941499 10081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:38.941545 10081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:38.941610 10081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:38.941640 10081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:38.950197 10081 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:39127
I20260811 11:55:38.950223 10220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:39127 every 8 connection(s)
I20260811 11:55:38.951032 10081 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:55:38.958350 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 10081
I20260811 11:55:38.958498 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance
I20260811 11:55:38.958839 6718 external_mini_cluster.cc:1428] Setting key 0ba5c559d0a4940947b046d94e9c5741
Aug 11 11:55:38 dist-test-slave-zkpd krb5kdc[9966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449338, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:38.964465 10055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:38.952828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:35553 (local address 127.6.143.190:46147)
0811 11:55:38.952972 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:38.952975 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:38.953764 (+ 789us) server_negotiation.cc:407] Connection header received
0811 11:55:38.955042 (+ 1278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:38.955045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:38.955092 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:38.955167 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:38.956271 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:38.956791 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:38.957880 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:38.958092 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:38.958166 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:38.960868 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:38.960871 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:38.960873 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:38.960896 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:38.962262 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:38.962817 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:38.962823 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:38.962824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:38.962926 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:38.963239 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:38.963244 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:38.963247 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:38.963497 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:38.963603 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:38.964082 (+ 479us) server_negotiation.cc:299] Negotiation successful
0811 11:55:38.964224 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 11:55:38.966033 10223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:38.953174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46147 (local address 127.6.143.129:35553)
0811 11:55:38.953607 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:38.953648 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:38.954846 (+ 1198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:38.955330 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:38.955336 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:38.955668 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:38.956151 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:38.956164 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:38.956893 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:38.956895 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:38.957778 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:38.957787 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:38.958241 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:38.958245 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:38.958456 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:38.959084 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:38.959105 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:38.960708 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:38.962387 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:38.962397 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:38.962414 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:38.962720 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:38.963021 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:38.963023 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:38.963024 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:38.963152 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:38.963597 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:38.963602 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:38.963815 (+ 213us) client_negotiation.cc:769] Sending connection context
0811 11:55:38.964018 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 11:55:38.964325 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":103,"threads_started":1}
I20260811 11:55:38.967242 10221 heartbeater.cc:347] Connected to a master server at 127.6.143.190:46147
I20260811 11:55:38.967483 10221 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:38.968014 10221 heartbeater.cc:511] Master 127.6.143.190:46147 requested a full tablet report, sending...
I20260811 11:55:38.969466 10014 ts_manager.cc:194] Registered new tserver with Master: ffa65a449c83433f8ac3c21bbc873189 (127.6.143.129:39127)
WARNING: no policy specified for kudu/127.6.143.130@KRBTEST.COM; defaulting to no policy
I20260811 11:55:38.970623 10014 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.129@KRBTEST.COM'} at 127.6.143.129:35553
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
I20260811 11:55:38.986508 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:0
--local_ip_for_outbound_sockets=127.6.143.130
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:46147
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:45051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:39.098206 10226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:39.098440 10226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:39.098480 10226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:39.102418 10226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:39.102465 10226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:39.102589 10226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:55:39.107520 10226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:45051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:46147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:39.108503 10226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:39.109473 10226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:39.119174 10231 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:39.119196 10232 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:39.119349 10236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:39.119233 10226 server_base.cc:1034] running on GCE node
I20260811 11:55:39.119834 10226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:39.120460 10226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:39.121616 10226 hybrid_clock.cc:648] HybridClock initialized: now 1786449339121601 us; error 27 us; skew 500 ppm
Aug 11 11:55:39 dist-test-slave-zkpd krb5kdc[9966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449339, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:39.124318 10226 init.cc:377] Logged in from keytab as kudu/127.6.143.130@KRBTEST.COM (short username kudu)
I20260811 11:55:39.125164 10226 webserver.cc:460] Webserver started at http://127.6.143.130:35945/ using document root <none> and password file <none>
I20260811 11:55:39.125778 10226 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:39.125841 10226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:39.126098 10226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:39.128269 10226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/data/instance:
uuid: "215a42aad9b14cd2ab1620592f673bdc"
format_stamp: "Formatted at 2026-08-11 11:55:39 on dist-test-slave-zkpd"
server_key: "2be8532cf468bcd4be6d1a62d0086122"
server_key_iv: "95d90cb89b52a40efb79d5f1266b8872"
server_key_version: "encryptionkey@0"
I20260811 11:55:39.128849 10226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance:
uuid: "215a42aad9b14cd2ab1620592f673bdc"
format_stamp: "Formatted at 2026-08-11 11:55:39 on dist-test-slave-zkpd"
server_key: "2be8532cf468bcd4be6d1a62d0086122"
server_key_iv: "95d90cb89b52a40efb79d5f1266b8872"
server_key_version: "encryptionkey@0"
I20260811 11:55:39.132449 10226 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 11:55:39.134680 10244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:39.135870 10226 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:55:39.136050 10226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "215a42aad9b14cd2ab1620592f673bdc"
format_stamp: "Formatted at 2026-08-11 11:55:39 on dist-test-slave-zkpd"
server_key: "2be8532cf468bcd4be6d1a62d0086122"
server_key_iv: "95d90cb89b52a40efb79d5f1266b8872"
server_key_version: "encryptionkey@0"
I20260811 11:55:39.136142 10226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:39.154767 10226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:39.157513 10226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:39.157712 10226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:39.158290 10226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:55:39.159112 10226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:39.159183 10226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:39.159257 10226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:39.159297 10226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:39.168190 10226 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:45109
I20260811 11:55:39.168215 10372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:45109 every 8 connection(s)
I20260811 11:55:39.169049 10226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
I20260811 11:55:39.172515 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 10226
I20260811 11:55:39.172650 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance
I20260811 11:55:39.172940 6718 external_mini_cluster.cc:1428] Setting key 01c27906de4296fe94473048fa224b08
Aug 11 11:55:39 dist-test-slave-zkpd krb5kdc[9966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449339, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.143.131@KRBTEST.COM; defaulting to no policy
I20260811 11:55:39.187217 10055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:39.170841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:33761 (local address 127.6.143.190:46147)
0811 11:55:39.170978 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:39.170982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:39.171764 (+ 782us) server_negotiation.cc:407] Connection header received
0811 11:55:39.173706 (+ 1942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:39.173709 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:39.173797 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:39.173888 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:39.176001 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:39.176865 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:39.178728 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:39.179294 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:39.179500 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:39.182782 (+ 3282us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:39.182785 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:39.182788 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:39.182816 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:39.184410 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:39.185302 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:39.185308 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:39.185326 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:39.185438 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:39.186074 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:39.186077 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:39.186079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:39.186267 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:39.186377 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:39.186906 (+ 529us) server_negotiation.cc:299] Negotiation successful
0811 11:55:39.187040 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":47}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.131@KRBTEST.COM" created.
I20260811 11:55:39.188247 10375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:39.171172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46147 (local address 127.6.143.130:33761)
0811 11:55:39.171611 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:39.171652 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:39.173122 (+ 1470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:39.174600 (+ 1478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:39.174618 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:39.175196 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:39.175784 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:39.175806 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:39.177073 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:39.177078 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:39.178578 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:39.178593 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:39.179529 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:39.179535 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:39.179743 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:39.180528 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:39.180554 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:39.182545 (+ 1991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:39.184527 (+ 1982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:39.184535 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:39.184551 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:39.185175 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:39.185728 (+ 553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:39.185734 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:39.185739 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:39.185970 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:39.186375 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:39.186382 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:39.186625 (+ 243us) client_negotiation.cc:769] Sending connection context
0811 11:55:39.186840 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 11:55:39.187168 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":110,"threads_started":1}
I20260811 11:55:39.190516 10373 heartbeater.cc:347] Connected to a master server at 127.6.143.190:46147
I20260811 11:55:39.190814 10373 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:39.191470 10373 heartbeater.cc:511] Master 127.6.143.190:46147 requested a full tablet report, sending...
I20260811 11:55:39.192732 10014 ts_manager.cc:194] Registered new tserver with Master: 215a42aad9b14cd2ab1620592f673bdc (127.6.143.130:45109)
I20260811 11:55:39.193627 10014 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.130@KRBTEST.COM'} at 127.6.143.130:33761
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
I20260811 11:55:39.205159 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:0
--local_ip_for_outbound_sockets=127.6.143.131
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:46147
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:45051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:39.322629 10378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:39.322866 10378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:39.322927 10378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:39.326877 10378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:39.326930 10378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:39.327035 10378 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:55:39.332372 10378 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:45051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:46147
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:39.333350 10378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:39.334353 10378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:39.343742 10383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:39.343775 10386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:39.343830 10378 server_base.cc:1034] running on GCE node
W20260811 11:55:39.343775 10384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:39.344430 10378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:39.345046 10378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:39.346225 10378 hybrid_clock.cc:648] HybridClock initialized: now 1786449339346207 us; error 27 us; skew 500 ppm
Aug 11 11:55:39 dist-test-slave-zkpd krb5kdc[9966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449339, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:39.349013 10378 init.cc:377] Logged in from keytab as kudu/127.6.143.131@KRBTEST.COM (short username kudu)
I20260811 11:55:39.349830 10378 webserver.cc:460] Webserver started at http://127.6.143.131:33413/ using document root <none> and password file <none>
I20260811 11:55:39.350389 10378 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:39.350476 10378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:39.350696 10378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:39.352766 10378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/data/instance:
uuid: "c0cd0cd69dd54cd287266f4fc01724bb"
format_stamp: "Formatted at 2026-08-11 11:55:39 on dist-test-slave-zkpd"
server_key: "65bc41261713e9c09187bb7881ff95c0"
server_key_iv: "6a06f7a2e52dad4e3fcf83f4c5431389"
server_key_version: "encryptionkey@0"
I20260811 11:55:39.353348 10378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance:
uuid: "c0cd0cd69dd54cd287266f4fc01724bb"
format_stamp: "Formatted at 2026-08-11 11:55:39 on dist-test-slave-zkpd"
server_key: "65bc41261713e9c09187bb7881ff95c0"
server_key_iv: "6a06f7a2e52dad4e3fcf83f4c5431389"
server_key_version: "encryptionkey@0"
I20260811 11:55:39.357035 10378 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260811 11:55:39.359474 10393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:39.360723 10378 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 11:55:39.360873 10378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "c0cd0cd69dd54cd287266f4fc01724bb"
format_stamp: "Formatted at 2026-08-11 11:55:39 on dist-test-slave-zkpd"
server_key: "65bc41261713e9c09187bb7881ff95c0"
server_key_iv: "6a06f7a2e52dad4e3fcf83f4c5431389"
server_key_version: "encryptionkey@0"
I20260811 11:55:39.360984 10378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:39.382810 10378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:39.386183 10378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:39.386405 10378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:39.387027 10378 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:55:39.387940 10378 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:39.388010 10378 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:39.388083 10378 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:39.388125 10378 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:39.396929 10378 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:41975
I20260811 11:55:39.396950 10522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:41975 every 8 connection(s)
I20260811 11:55:39.397840 10378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:55:39.402437 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 10378
I20260811 11:55:39.402532 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance
I20260811 11:55:39.402817 6718 external_mini_cluster.cc:1428] Setting key 4f966b0c3d39c3eabbad9152abd5bfea
Aug 11 11:55:39 dist-test-slave-zkpd krb5kdc[9966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449339, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:39.411228 10055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:39.399620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:38443 (local address 127.6.143.190:46147)
0811 11:55:39.399765 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:39.399770 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:39.400710 (+ 940us) server_negotiation.cc:407] Connection header received
0811 11:55:39.401991 (+ 1281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:39.401994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:39.402042 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:39.402116 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:39.403522 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:39.404113 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:39.405359 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:39.405581 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:39.405689 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:39.408011 (+ 2322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:39.408019 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:39.408021 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:39.408045 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:39.409233 (+ 1188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:39.409743 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:39.409746 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:39.409748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:39.409828 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:39.410164 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:39.410168 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:39.410170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:39.410343 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:39.410448 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:39.410947 (+ 499us) server_negotiation.cc:299] Negotiation successful
0811 11:55:39.411064 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 11:55:39.412112 10525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:39.400088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46147 (local address 127.6.143.131:38443)
0811 11:55:39.400550 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:39.400593 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:39.401798 (+ 1205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:39.402289 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:39.402295 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:39.402738 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:39.403378 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:39.403396 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:39.404236 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:39.404240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:39.405237 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:39.405247 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:39.405727 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:39.405731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:39.405912 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:39.406435 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:39.406451 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:39.407816 (+ 1365us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:39.409370 (+ 1554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:39.409376 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:39.409390 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:39.409639 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:39.409927 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:39.409931 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:39.409933 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:39.410069 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:39.410447 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:39.410453 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:39.410696 (+ 243us) client_negotiation.cc:769] Sending connection context
0811 11:55:39.410899 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 11:55:39.411202 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":113,"threads_started":1}
I20260811 11:55:39.413216 10523 heartbeater.cc:347] Connected to a master server at 127.6.143.190:46147
I20260811 11:55:39.413431 10523 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:39.414014 10523 heartbeater.cc:511] Master 127.6.143.190:46147 requested a full tablet report, sending...
I20260811 11:55:39.414963 10014 ts_manager.cc:194] Registered new tserver with Master: c0cd0cd69dd54cd287266f4fc01724bb (127.6.143.131:41975)
I20260811 11:55:39.415720 10014 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.131@KRBTEST.COM'} at 127.6.143.131:38443
I20260811 11:55:39.416538 6718 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 11:55:39 dist-test-slave-zkpd krb5kdc[9966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449339, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 11:55:39 dist-test-slave-zkpd krb5kdc[9966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449339, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:39.442251 10055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:39.434239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59258 (local address 127.6.143.190:46147)
0811 11:55:39.434361 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:39.434365 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:39.434547 (+ 182us) server_negotiation.cc:407] Connection header received
0811 11:55:39.434663 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:39.434666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:39.434707 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:39.434778 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:39.435439 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:39.436004 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:39.436792 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:39.436991 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:39.437063 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:39.439295 (+ 2232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:39.439298 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:39.439300 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:39.439322 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:39.440895 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:39.441262 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:39.441265 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:39.441266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:39.441336 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:39.441555 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:39.441557 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:39.441558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:39.441754 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:39.441857 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:39.442046 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 11:55:39.442135 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 11:55:39.442611 10014 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59258
I20260811 11:55:39.443382 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 10081
I20260811 11:55:39.451874 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 10226
I20260811 11:55:39.460415 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 10378
I20260811 11:55:39.474256 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 9980
2026-08-11T11:55:39Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2174 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:55:39 dist-test-slave-zkpd krb5kdc[10536](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:55:39 dist-test-slave-zkpd krb5kdc[10536](info): set up 2 sockets
Aug 11 11:55:39 dist-test-slave-zkpd krb5kdc[10536](info): commencing operation
krb5kdc: starting...
W20260811 11:55:40.556622 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:55:40 dist-test-slave-zkpd krb5kdc[10536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:55:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:55:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
I20260811 11:55:40.676869 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:40619
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:40619
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:40.787689 10550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:40.787981 10550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:40.788062 10550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:40.792011 10550 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:55:40.792097 10550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:40.792136 10550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:40.792187 10550 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:55:40.792227 10550 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:55:40.797390 10550 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:40619
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:40619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:40.798435 10550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:40.799443 10550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:40.808668 10559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:40.808732 10556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:40.808701 10557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:40.808979 10550 server_base.cc:1034] running on GCE node
I20260811 11:55:40.809536 10550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:40.810572 10550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:40.811750 10550 hybrid_clock.cc:648] HybridClock initialized: now 1786449340811732 us; error 38 us; skew 500 ppm
Aug 11 11:55:40 dist-test-slave-zkpd krb5kdc[10536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449340, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:40.818709 10550 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:55:40.819585 10550 webserver.cc:460] Webserver started at http://127.6.143.190:44849/ using document root <none> and password file <none>
I20260811 11:55:40.820180 10550 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:40.820242 10550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:40.820410 10550 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:40.822381 10550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "444144d63ece43a38f123e3cae4861e0"
format_stamp: "Formatted at 2026-08-11 11:55:40 on dist-test-slave-zkpd"
server_key: "6ca0641fdaa3fc2365752df622b2b025"
server_key_iv: "bc583237935b205ec49c10ed6a1922bc"
server_key_version: "encryptionkey@0"
I20260811 11:55:40.822902 10550 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "444144d63ece43a38f123e3cae4861e0"
format_stamp: "Formatted at 2026-08-11 11:55:40 on dist-test-slave-zkpd"
server_key: "6ca0641fdaa3fc2365752df622b2b025"
server_key_iv: "bc583237935b205ec49c10ed6a1922bc"
server_key_version: "encryptionkey@0"
I20260811 11:55:40.826520 10550 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 11:55:40.828805 10566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:40.829973 10550 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:55:40.830073 10550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "444144d63ece43a38f123e3cae4861e0"
format_stamp: "Formatted at 2026-08-11 11:55:40 on dist-test-slave-zkpd"
server_key: "6ca0641fdaa3fc2365752df622b2b025"
server_key_iv: "bc583237935b205ec49c10ed6a1922bc"
server_key_version: "encryptionkey@0"
I20260811 11:55:40.830152 10550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:40.850121 10550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:40.852851 10550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:40.853013 10550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:40.860131 10550 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:40619
I20260811 11:55:40.860157 10622 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:40619 every 8 connection(s)
I20260811 11:55:40.861092 10550 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:55:40.862917 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 10550
I20260811 11:55:40.863039 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:55:40.863327 6718 external_mini_cluster.cc:1428] Setting key 468a4e35f089d6094f5f07dc08989a0f
I20260811 11:55:40.864239 10623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:40.870000 10623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 444144d63ece43a38f123e3cae4861e0: Bootstrap starting.
Aug 11 11:55:40 dist-test-slave-zkpd krb5kdc[10536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:40.872325 10623 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 444144d63ece43a38f123e3cae4861e0: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:40.873275 10623 log.cc:824] T 00000000000000000000000000000000 P 444144d63ece43a38f123e3cae4861e0: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:40.875530 10623 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 444144d63ece43a38f123e3cae4861e0: No bootstrap required, opened a new log
I20260811 11:55:40.877566 10626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:40.864754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47068 (local address 127.6.143.190:40619)
0811 11:55:40.865228 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:40.865243 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:40.865279 (+ 36us) server_negotiation.cc:407] Connection header received
0811 11:55:40.865942 (+ 663us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:40.865959 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:40.866277 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:40.866624 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:40.867454 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:40.868544 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:40.869433 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:40.869904 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:40.870073 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:40.872582 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:40.872590 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:40.872610 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:40.872646 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:40.874490 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:40.874934 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:40.874940 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:40.874946 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:40.875082 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:40.875339 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:40.875342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:40.875344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:40.875830 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:40.876093 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:40.876332 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 11:55:40.876626 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":171,"threads_started":1}
I20260811 11:55:40.879894 10623 raft_consensus.cc:348] T 00000000000000000000000000000000 P 444144d63ece43a38f123e3cae4861e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "444144d63ece43a38f123e3cae4861e0" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 40619 } }
I20260811 11:55:40.880137 10623 raft_consensus.cc:374] T 00000000000000000000000000000000 P 444144d63ece43a38f123e3cae4861e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:40.880180 10623 raft_consensus.cc:729] T 00000000000000000000000000000000 P 444144d63ece43a38f123e3cae4861e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 444144d63ece43a38f123e3cae4861e0, State: Initialized, Role: FOLLOWER
I20260811 11:55:40.880549 10623 consensus_queue.cc:260] T 00000000000000000000000000000000 P 444144d63ece43a38f123e3cae4861e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "444144d63ece43a38f123e3cae4861e0" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 40619 } }
I20260811 11:55:40.880656 10623 raft_consensus.cc:388] T 00000000000000000000000000000000 P 444144d63ece43a38f123e3cae4861e0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:55:40.880700 10623 raft_consensus.cc:482] T 00000000000000000000000000000000 P 444144d63ece43a38f123e3cae4861e0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:55:40.880767 10623 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 444144d63ece43a38f123e3cae4861e0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:40.881603 10623 raft_consensus.cc:504] T 00000000000000000000000000000000 P 444144d63ece43a38f123e3cae4861e0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "444144d63ece43a38f123e3cae4861e0" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 40619 } }
I20260811 11:55:40.881897 10623 leader_election.cc:302] T 00000000000000000000000000000000 P 444144d63ece43a38f123e3cae4861e0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 444144d63ece43a38f123e3cae4861e0; no voters:
I20260811 11:55:40.882176 10623 leader_election.cc:288] T 00000000000000000000000000000000 P 444144d63ece43a38f123e3cae4861e0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:55:40.882249 10628 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 444144d63ece43a38f123e3cae4861e0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:40.882494 10628 raft_consensus.cc:686] T 00000000000000000000000000000000 P 444144d63ece43a38f123e3cae4861e0 [term 1 LEADER]: Becoming Leader. State: Replica: 444144d63ece43a38f123e3cae4861e0, State: Running, Role: LEADER
I20260811 11:55:40.882838 10628 consensus_queue.cc:237] T 00000000000000000000000000000000 P 444144d63ece43a38f123e3cae4861e0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "444144d63ece43a38f123e3cae4861e0" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 40619 } }
I20260811 11:55:40.883455 10623 sys_catalog.cc:564] T 00000000000000000000000000000000 P 444144d63ece43a38f123e3cae4861e0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:55:40.884049 10628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 444144d63ece43a38f123e3cae4861e0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 444144d63ece43a38f123e3cae4861e0. Latest consensus state: current_term: 1 leader_uuid: "444144d63ece43a38f123e3cae4861e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "444144d63ece43a38f123e3cae4861e0" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 40619 } } }
I20260811 11:55:40.884256 10628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 444144d63ece43a38f123e3cae4861e0 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:40.884447 10630 sys_catalog.cc:455] T 00000000000000000000000000000000 P 444144d63ece43a38f123e3cae4861e0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "444144d63ece43a38f123e3cae4861e0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "444144d63ece43a38f123e3cae4861e0" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 40619 } } }
I20260811 11:55:40.884624 10630 sys_catalog.cc:458] T 00000000000000000000000000000000 P 444144d63ece43a38f123e3cae4861e0 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:40.884713 10638 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:55:40.888067 10638 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:55:40.893702 10638 catalog_manager.cc:1347] Generated new cluster ID: 4c87eda386c1497a8959eddc009f6098
I20260811 11:55:40.893770 10638 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:55:40.898687 10638 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:55:40.899528 10638 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:55:40.905088 10638 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 444144d63ece43a38f123e3cae4861e0: Generated new TSK 0
I20260811 11:55:40.905789 10638 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
I20260811 11:55:40.943008 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:0
--local_ip_for_outbound_sockets=127.6.143.129
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:40619
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:41.054937 10652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:41.055186 10652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:41.055226 10652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:41.059213 10652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:41.059264 10652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:41.059360 10652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:55:41.064425 10652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:40619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:41.065392 10652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:41.066416 10652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:41.075831 10658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:41.075841 10657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:41.075990 10652 server_base.cc:1034] running on GCE node
W20260811 11:55:41.075841 10660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:41.076509 10652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:41.077157 10652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:41.078330 10652 hybrid_clock.cc:648] HybridClock initialized: now 1786449341078315 us; error 28 us; skew 500 ppm
Aug 11 11:55:41 dist-test-slave-zkpd krb5kdc[10536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449341, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:41.081080 10652 init.cc:377] Logged in from keytab as kudu/127.6.143.129@KRBTEST.COM (short username kudu)
I20260811 11:55:41.081928 10652 webserver.cc:460] Webserver started at http://127.6.143.129:42773/ using document root <none> and password file <none>
I20260811 11:55:41.082487 10652 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:41.082547 10652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:41.082799 10652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:41.084909 10652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-0/data/instance:
uuid: "7bc6ffb21c5840f5864afeba7baceb60"
format_stamp: "Formatted at 2026-08-11 11:55:41 on dist-test-slave-zkpd"
server_key: "fbc037b58d9f0f4898d480f8cdff238b"
server_key_iv: "5b39676f669bd17d34d16adb4c8b11eb"
server_key_version: "encryptionkey@0"
I20260811 11:55:41.085464 10652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance:
uuid: "7bc6ffb21c5840f5864afeba7baceb60"
format_stamp: "Formatted at 2026-08-11 11:55:41 on dist-test-slave-zkpd"
server_key: "fbc037b58d9f0f4898d480f8cdff238b"
server_key_iv: "5b39676f669bd17d34d16adb4c8b11eb"
server_key_version: "encryptionkey@0"
I20260811 11:55:41.089304 10652 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 11:55:41.091631 10668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:41.092864 10652 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:55:41.092984 10652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "7bc6ffb21c5840f5864afeba7baceb60"
format_stamp: "Formatted at 2026-08-11 11:55:41 on dist-test-slave-zkpd"
server_key: "fbc037b58d9f0f4898d480f8cdff238b"
server_key_iv: "5b39676f669bd17d34d16adb4c8b11eb"
server_key_version: "encryptionkey@0"
I20260811 11:55:41.093080 10652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:41.116251 10652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:41.119136 10652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:41.119330 10652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:41.119975 10652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:55:41.120805 10652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:41.120852 10652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:41.120918 10652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:41.120949 10652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:41.129570 10652 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:43675
I20260811 11:55:41.129597 10791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:43675 every 8 connection(s)
I20260811 11:55:41.130447 10652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:55:41.139286 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 10652
I20260811 11:55:41.139421 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance
Aug 11 11:55:41 dist-test-slave-zkpd krb5kdc[10536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449341, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:41.139727 6718 external_mini_cluster.cc:1428] Setting key d1ea1d9fa7b52562b2feaad2e7d509a1
I20260811 11:55:41.145874 10626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:41.132241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:58463 (local address 127.6.143.190:40619)
0811 11:55:41.132394 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:41.132399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:41.133220 (+ 821us) server_negotiation.cc:407] Connection header received
0811 11:55:41.134691 (+ 1471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:41.134694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:41.134740 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:41.134814 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:41.135969 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:41.136494 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:41.137565 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:41.137785 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:41.137862 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:41.140271 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:41.140278 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:41.140284 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:41.140319 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:41.143644 (+ 3325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:41.144295 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:41.144298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:41.144299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:41.144379 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:41.144762 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:41.144764 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:41.144766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:41.144945 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:41.145022 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:41.145644 (+ 622us) server_negotiation.cc:299] Negotiation successful
0811 11:55:41.145745 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 11:55:41.147109 10794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:41.132614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:40619 (local address 127.6.143.129:58463)
0811 11:55:41.133083 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:41.133128 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:41.134492 (+ 1364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:41.134971 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:41.134978 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:41.135327 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:41.135784 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:41.135798 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:41.136610 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:41.136612 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:41.137463 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:41.137472 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:41.137906 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:41.137909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:41.138044 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:41.138541 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:41.138557 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:41.140112 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:41.143770 (+ 3658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:41.143777 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:41.143794 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:41.144197 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:41.144472 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:41.144478 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:41.144483 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:41.144670 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:41.145050 (+ 380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:41.145061 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:41.145374 (+ 313us) client_negotiation.cc:769] Sending connection context
0811 11:55:41.145598 (+ 224us) client_negotiation.cc:240] Negotiation successful
0811 11:55:41.145966 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":102,"threads_started":1}
I20260811 11:55:41.148545 10792 heartbeater.cc:347] Connected to a master server at 127.6.143.190:40619
I20260811 11:55:41.148828 10792 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:41.149338 10792 heartbeater.cc:511] Master 127.6.143.190:40619 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.143.130@KRBTEST.COM; defaulting to no policy
I20260811 11:55:41.150995 10584 ts_manager.cc:194] Registered new tserver with Master: 7bc6ffb21c5840f5864afeba7baceb60 (127.6.143.129:43675)
I20260811 11:55:41.152379 10584 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.129@KRBTEST.COM'} at 127.6.143.129:58463
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
I20260811 11:55:41.168192 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:0
--local_ip_for_outbound_sockets=127.6.143.130
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:40619
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:41.279676 10798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:41.279966 10798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:41.280047 10798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:41.284209 10798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:41.284271 10798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:41.284375 10798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:55:41.289631 10798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:40619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:41.290632 10798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:41.291710 10798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:41.301297 10808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:41.301357 10803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:41.301441 10798 server_base.cc:1034] running on GCE node
W20260811 11:55:41.301363 10806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:41.302040 10798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:41.302654 10798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:41.303833 10798 hybrid_clock.cc:648] HybridClock initialized: now 1786449341303806 us; error 45 us; skew 500 ppm
Aug 11 11:55:41 dist-test-slave-zkpd krb5kdc[10536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449341, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:41.306491 10798 init.cc:377] Logged in from keytab as kudu/127.6.143.130@KRBTEST.COM (short username kudu)
I20260811 11:55:41.307811 10798 webserver.cc:460] Webserver started at http://127.6.143.130:39281/ using document root <none> and password file <none>
I20260811 11:55:41.308377 10798 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:41.308441 10798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:41.308599 10798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:41.310488 10798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-1/data/instance:
uuid: "ab44c9452f934b1792ce4a46064a2d72"
format_stamp: "Formatted at 2026-08-11 11:55:41 on dist-test-slave-zkpd"
server_key: "3ac6de019f977e4bcbaf0c949b01ab6a"
server_key_iv: "955f18a0ff102b33a2df7c8f9db5713e"
server_key_version: "encryptionkey@0"
I20260811 11:55:41.311005 10798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance:
uuid: "ab44c9452f934b1792ce4a46064a2d72"
format_stamp: "Formatted at 2026-08-11 11:55:41 on dist-test-slave-zkpd"
server_key: "3ac6de019f977e4bcbaf0c949b01ab6a"
server_key_iv: "955f18a0ff102b33a2df7c8f9db5713e"
server_key_version: "encryptionkey@0"
I20260811 11:55:41.314579 10798 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 11:55:41.316877 10816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:41.318071 10798 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:55:41.318198 10798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "ab44c9452f934b1792ce4a46064a2d72"
format_stamp: "Formatted at 2026-08-11 11:55:41 on dist-test-slave-zkpd"
server_key: "3ac6de019f977e4bcbaf0c949b01ab6a"
server_key_iv: "955f18a0ff102b33a2df7c8f9db5713e"
server_key_version: "encryptionkey@0"
I20260811 11:55:41.318295 10798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:41.329370 10798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:41.332624 10798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:41.332816 10798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:41.333387 10798 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:55:41.334254 10798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:41.334301 10798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:41.334340 10798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:41.334362 10798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:41.343174 10798 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:43975
I20260811 11:55:41.343232 10934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:43975 every 8 connection(s)
I20260811 11:55:41.344259 10798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
I20260811 11:55:41.344458 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 10798
I20260811 11:55:41.344579 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance
I20260811 11:55:41.344946 6718 external_mini_cluster.cc:1428] Setting key 10ecf42bb5bd5461e18526beb12b8140
Aug 11 11:55:41 dist-test-slave-zkpd krb5kdc[10536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449341, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.143.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.131@KRBTEST.COM" created.
I20260811 11:55:41.358615 10626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:41.346160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:58995 (local address 127.6.143.190:40619)
0811 11:55:41.346293 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:41.346299 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:41.347308 (+ 1009us) server_negotiation.cc:407] Connection header received
0811 11:55:41.348830 (+ 1522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:41.348833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:41.348879 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:41.348952 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:41.350476 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:41.350999 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:41.352245 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:41.352475 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:41.352583 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:41.354887 (+ 2304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:41.354890 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:41.354892 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:41.354914 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:41.356458 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:41.356959 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:41.356962 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:41.356964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:41.357047 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:41.357459 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:41.357463 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:41.357464 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:41.357707 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:41.357819 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:41.358383 (+ 564us) server_negotiation.cc:299] Negotiation successful
0811 11:55:41.358495 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 11:55:41.359730 10937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:41.346639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:40619 (local address 127.6.143.130:58995)
0811 11:55:41.347164 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:41.347204 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:41.348585 (+ 1381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:41.349132 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:41.349143 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:41.349649 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:41.350329 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:41.350346 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:41.351124 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:41.351128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:41.352125 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:41.352136 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:41.352619 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:41.352623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:41.352786 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:41.353332 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:41.353348 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:41.354717 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:41.356583 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:41.356589 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:41.356602 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:41.356847 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:41.357169 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:41.357175 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:41.357180 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:41.357351 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:41.357817 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:41.357824 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:41.358088 (+ 264us) client_negotiation.cc:769] Sending connection context
0811 11:55:41.358310 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 11:55:41.358686 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":170,"threads_started":1}
I20260811 11:55:41.360888 10935 heartbeater.cc:347] Connected to a master server at 127.6.143.190:40619
I20260811 11:55:41.361148 10935 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:41.361763 10935 heartbeater.cc:511] Master 127.6.143.190:40619 requested a full tablet report, sending...
I20260811 11:55:41.362954 10584 ts_manager.cc:194] Registered new tserver with Master: ab44c9452f934b1792ce4a46064a2d72 (127.6.143.130:43975)
I20260811 11:55:41.363823 10584 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.130@KRBTEST.COM'} at 127.6.143.130:58995
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
I20260811 11:55:41.372293 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:0
--local_ip_for_outbound_sockets=127.6.143.131
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:40619
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:40041
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:41.485116 10941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:41.485370 10941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:41.485445 10941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:41.489662 10941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:41.489714 10941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:41.489769 10941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:55:41.494827 10941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:40041
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:40619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10941
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:41.495752 10941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:41.496806 10941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:41.506248 10949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:41.506315 10941 server_base.cc:1034] running on GCE node
W20260811 11:55:41.506244 10947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:41.506245 10946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:41.506946 10941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:41.507532 10941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:41.508687 10941 hybrid_clock.cc:648] HybridClock initialized: now 1786449341508671 us; error 27 us; skew 500 ppm
Aug 11 11:55:41 dist-test-slave-zkpd krb5kdc[10536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449341, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:41.511484 10941 init.cc:377] Logged in from keytab as kudu/127.6.143.131@KRBTEST.COM (short username kudu)
I20260811 11:55:41.512351 10941 webserver.cc:460] Webserver started at http://127.6.143.131:33623/ using document root <none> and password file <none>
I20260811 11:55:41.512915 10941 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:41.513003 10941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:41.513227 10941 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:41.515237 10941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-2/data/instance:
uuid: "b9c936b3982f4f269179e895f098c945"
format_stamp: "Formatted at 2026-08-11 11:55:41 on dist-test-slave-zkpd"
server_key: "ed4cbf74e8007377d05368fbe593c9c8"
server_key_iv: "603a82d566b3baf24f0c1a9884e42e81"
server_key_version: "encryptionkey@0"
I20260811 11:55:41.515807 10941 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance:
uuid: "b9c936b3982f4f269179e895f098c945"
format_stamp: "Formatted at 2026-08-11 11:55:41 on dist-test-slave-zkpd"
server_key: "ed4cbf74e8007377d05368fbe593c9c8"
server_key_iv: "603a82d566b3baf24f0c1a9884e42e81"
server_key_version: "encryptionkey@0"
I20260811 11:55:41.519578 10941 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 11:55:41.522008 10956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:41.523222 10941 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:41.523347 10941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "b9c936b3982f4f269179e895f098c945"
format_stamp: "Formatted at 2026-08-11 11:55:41 on dist-test-slave-zkpd"
server_key: "ed4cbf74e8007377d05368fbe593c9c8"
server_key_iv: "603a82d566b3baf24f0c1a9884e42e81"
server_key_version: "encryptionkey@0"
I20260811 11:55:41.523460 10941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:41.546069 10941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:41.549259 10941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:41.549464 10941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:41.550035 10941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:55:41.550876 10941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:41.550942 10941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:41.551013 10941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:41.551052 10941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:41.559805 10941 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:32843
I20260811 11:55:41.559868 11081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:32843 every 8 connection(s)
I20260811 11:55:41.560711 10941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:55:41.568717 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 10941
I20260811 11:55:41.568852 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance
I20260811 11:55:41.569209 6718 external_mini_cluster.cc:1428] Setting key c766955ec22a595dfa7942d1cfb9e3e2
Aug 11 11:55:41 dist-test-slave-zkpd krb5kdc[10536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449341, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:41.574360 10626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:41.562476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:41095 (local address 127.6.143.190:40619)
0811 11:55:41.562621 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:41.562624 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:41.563441 (+ 817us) server_negotiation.cc:407] Connection header received
0811 11:55:41.564848 (+ 1407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:41.564851 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:41.564914 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:41.565020 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:41.566178 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:41.566710 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:41.567828 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:41.568106 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:41.568218 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:41.570912 (+ 2694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:41.570915 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:41.570918 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:41.570950 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:41.572371 (+ 1421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:41.572879 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:41.572883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:41.572885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:41.572967 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:41.573301 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:41.573306 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:41.573307 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:41.573493 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:41.573598 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:41.574081 (+ 483us) server_negotiation.cc:299] Negotiation successful
0811 11:55:41.574196 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 11:55:41.575107 11084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:41.562834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:40619 (local address 127.6.143.131:41095)
0811 11:55:41.563282 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:41.563323 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:41.564632 (+ 1309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:41.565199 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:41.565206 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:41.565548 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:41.566025 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:41.566039 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:41.566836 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:41.566840 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:41.567697 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:41.567707 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:41.568260 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:41.568264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:41.568483 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:41.569157 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:41.569179 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:41.570747 (+ 1568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:41.572502 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:41.572508 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:41.572522 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:41.572774 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:41.573066 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:41.573069 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:41.573071 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:41.573206 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:41.573597 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:41.573604 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:41.573846 (+ 242us) client_negotiation.cc:769] Sending connection context
0811 11:55:41.574037 (+ 191us) client_negotiation.cc:240] Negotiation successful
0811 11:55:41.574340 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":114,"threads_started":1}
I20260811 11:55:41.576157 11082 heartbeater.cc:347] Connected to a master server at 127.6.143.190:40619
I20260811 11:55:41.576364 11082 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:41.576783 11082 heartbeater.cc:511] Master 127.6.143.190:40619 requested a full tablet report, sending...
I20260811 11:55:41.577699 10584 ts_manager.cc:194] Registered new tserver with Master: b9c936b3982f4f269179e895f098c945 (127.6.143.131:32843)
I20260811 11:55:41.578205 10584 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.131@KRBTEST.COM'} at 127.6.143.131:41095
I20260811 11:55:41.582759 6718 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 11:55:41 dist-test-slave-zkpd krb5kdc[10536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 11:55:41 dist-test-slave-zkpd krb5kdc[10536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.129@KRBTEST.COM
I20260811 11:55:41.609143 11089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:41.600360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44070 (local address 127.6.143.129:43675)
0811 11:55:41.600637 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:41.600651 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:41.600673 (+ 22us) server_negotiation.cc:407] Connection header received
0811 11:55:41.600721 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:41.600726 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:41.600908 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:41.601028 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:41.601657 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:41.602402 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:41.603163 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:41.603504 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:41.603587 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:41.606061 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:41.606070 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:41.606083 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:41.606105 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:41.607479 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:41.607874 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:41.607879 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:41.607880 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:41.608009 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:41.608244 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:41.608246 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:41.608247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:41.608469 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:41.608614 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:41.608783 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 11:55:41.608954 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":86,"threads_started":1}
Aug 11 11:55:41 dist-test-slave-zkpd krb5kdc[10536](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449341, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:41.619150 10626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:41.611835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47078 (local address 127.6.143.190:40619)
0811 11:55:41.612004 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:41.612008 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:41.612048 (+ 40us) server_negotiation.cc:407] Connection header received
0811 11:55:41.612159 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:41.612160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:41.612200 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:41.612266 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:41.612778 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:41.613266 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:41.613972 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:41.614169 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:41.614241 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:41.616377 (+ 2136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:41.616381 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:41.616383 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:41.616405 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:41.617758 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:41.618097 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:41.618100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:41.618101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:41.618171 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:41.618383 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:41.618385 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:41.618386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:41.618574 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:41.618670 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:41.618846 (+ 176us) server_negotiation.cc:299] Negotiation successful
0811 11:55:41.618970 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 11:55:41.619531 10584 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47078
I20260811 11:55:41.620155 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 10652
I20260811 11:55:41.633028 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 10798
I20260811 11:55:41.640174 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 10941
I20260811 11:55:41.646864 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 10550
2026-08-11T11:55:41Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2172 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:55:41 dist-test-slave-zkpd krb5kdc[11095](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:55:41 dist-test-slave-zkpd krb5kdc[11095](info): set up 2 sockets
Aug 11 11:55:41 dist-test-slave-zkpd krb5kdc[11095](info): commencing operation
krb5kdc: starting...
W20260811 11:55:42.720867 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:55:42 dist-test-slave-zkpd krb5kdc[11095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:55:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:55:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
I20260811 11:55:42.844316 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:38681
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:39559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:38681
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:42.955658 11110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:42.955991 11110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:42.956077 11110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:42.960034 11110 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:55:42.960103 11110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:42.960139 11110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:42.960175 11110 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:55:42.960209 11110 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:55:42.965303 11110 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:39559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:38681
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:38681
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:42.966298 11110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:42.967294 11110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:42.975548 11115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:42.975523 11118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:42.975529 11116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:42.975529 11110 server_base.cc:1034] running on GCE node
I20260811 11:55:42.976307 11110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:42.977342 11110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:42.978507 11110 hybrid_clock.cc:648] HybridClock initialized: now 1786449342978487 us; error 31 us; skew 500 ppm
Aug 11 11:55:42 dist-test-slave-zkpd krb5kdc[11095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449342, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:42.981225 11110 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:55:42.981971 11110 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:42.982035 11110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:42.982280 11110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:42.984342 11110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "8549bae94c4b480a815785e214edf37b"
format_stamp: "Formatted at 2026-08-11 11:55:42 on dist-test-slave-zkpd"
server_key: "42c1d9623e59f60e4c196c363c94fd12"
server_key_iv: "3d333ca4bed8f3bdfe65113a3bf3369a"
server_key_version: "encryptionkey@0"
I20260811 11:55:42.984905 11110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "8549bae94c4b480a815785e214edf37b"
format_stamp: "Formatted at 2026-08-11 11:55:42 on dist-test-slave-zkpd"
server_key: "42c1d9623e59f60e4c196c363c94fd12"
server_key_iv: "3d333ca4bed8f3bdfe65113a3bf3369a"
server_key_version: "encryptionkey@0"
I20260811 11:55:42.988472 11110 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 11:55:42.990736 11124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:42.992031 11110 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:55:42.992149 11110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "8549bae94c4b480a815785e214edf37b"
format_stamp: "Formatted at 2026-08-11 11:55:42 on dist-test-slave-zkpd"
server_key: "42c1d9623e59f60e4c196c363c94fd12"
server_key_iv: "3d333ca4bed8f3bdfe65113a3bf3369a"
server_key_version: "encryptionkey@0"
I20260811 11:55:42.992244 11110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:43.021922 11110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:43.024755 11110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:43.024964 11110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:43.031993 11110 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:38681
I20260811 11:55:43.032255 11183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:38681 every 8 connection(s)
I20260811 11:55:43.032673 11110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:55:43.035391 11184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:43.040863 11184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8549bae94c4b480a815785e214edf37b: Bootstrap starting.
I20260811 11:55:43.041055 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 11110
I20260811 11:55:43.041183 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:55:43.041487 6718 external_mini_cluster.cc:1428] Setting key 68ebf3481473dc246633461c16bed738
I20260811 11:55:43.043579 11184 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8549bae94c4b480a815785e214edf37b: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:43.044377 11184 log.cc:824] T 00000000000000000000000000000000 P 8549bae94c4b480a815785e214edf37b: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:43.046187 11184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8549bae94c4b480a815785e214edf37b: No bootstrap required, opened a new log
I20260811 11:55:43.049787 11184 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8549bae94c4b480a815785e214edf37b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8549bae94c4b480a815785e214edf37b" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38681 } }
I20260811 11:55:43.050074 11184 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8549bae94c4b480a815785e214edf37b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:43.050135 11184 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8549bae94c4b480a815785e214edf37b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8549bae94c4b480a815785e214edf37b, State: Initialized, Role: FOLLOWER
Aug 11 11:55:43 dist-test-slave-zkpd krb5kdc[11095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449342, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:43.050565 11184 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8549bae94c4b480a815785e214edf37b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8549bae94c4b480a815785e214edf37b" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38681 } }
I20260811 11:55:43.050715 11184 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8549bae94c4b480a815785e214edf37b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:55:43.050782 11184 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8549bae94c4b480a815785e214edf37b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:55:43.050885 11184 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8549bae94c4b480a815785e214edf37b [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:43.052038 11184 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8549bae94c4b480a815785e214edf37b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8549bae94c4b480a815785e214edf37b" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38681 } }
I20260811 11:55:43.052373 11184 leader_election.cc:302] T 00000000000000000000000000000000 P 8549bae94c4b480a815785e214edf37b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8549bae94c4b480a815785e214edf37b; no voters:
I20260811 11:55:43.052670 11184 leader_election.cc:288] T 00000000000000000000000000000000 P 8549bae94c4b480a815785e214edf37b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:55:43.052783 11189 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8549bae94c4b480a815785e214edf37b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:43.053015 11189 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8549bae94c4b480a815785e214edf37b [term 1 LEADER]: Becoming Leader. State: Replica: 8549bae94c4b480a815785e214edf37b, State: Running, Role: LEADER
I20260811 11:55:43.053429 11189 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8549bae94c4b480a815785e214edf37b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8549bae94c4b480a815785e214edf37b" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38681 } }
I20260811 11:55:43.054028 11184 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8549bae94c4b480a815785e214edf37b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:55:43.054701 11191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8549bae94c4b480a815785e214edf37b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8549bae94c4b480a815785e214edf37b. Latest consensus state: current_term: 1 leader_uuid: "8549bae94c4b480a815785e214edf37b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8549bae94c4b480a815785e214edf37b" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38681 } } }
I20260811 11:55:43.054939 11191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8549bae94c4b480a815785e214edf37b [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:43.055351 11190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8549bae94c4b480a815785e214edf37b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8549bae94c4b480a815785e214edf37b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8549bae94c4b480a815785e214edf37b" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38681 } } }
I20260811 11:55:43.055470 11200 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:55:43.056157 11190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8549bae94c4b480a815785e214edf37b [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:43.056775 11187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:43.042792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51056 (local address 127.6.143.190:38681)
0811 11:55:43.043213 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:43.043228 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:43.043262 (+ 34us) server_negotiation.cc:407] Connection header received
0811 11:55:43.044003 (+ 741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:43.044023 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:43.044350 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:43.044713 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:43.045541 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.046547 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.047414 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.047932 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.048100 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:43.050807 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:43.050816 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:43.050836 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:43.050876 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:43.052850 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:43.053394 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:43.053399 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:43.053405 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:43.053567 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:43.053926 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:43.053929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:43.053931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:43.054415 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:43.054614 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:43.055286 (+ 672us) server_negotiation.cc:299] Negotiation successful
0811 11:55:43.055632 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":125,"threads_started":1}
I20260811 11:55:43.058710 11200 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:55:43.064010 11200 catalog_manager.cc:1347] Generated new cluster ID: 3c01534989984d78bfc4be819ae35af8
I20260811 11:55:43.064071 11200 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:55:43.077353 11200 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:55:43.078182 11200 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:55:43.085611 11200 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8549bae94c4b480a815785e214edf37b: Generated new TSK 0
I20260811 11:55:43.086237 11200 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
I20260811 11:55:43.118217 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:0
--local_ip_for_outbound_sockets=127.6.143.129
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:38681
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:39559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:43.232237 11212 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:43.232476 11212 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:43.232546 11212 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:43.236869 11212 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:43.236927 11212 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:43.237051 11212 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:55:43.242194 11212 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:39559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:38681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11212
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:43.243181 11212 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:43.244266 11212 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:43.253381 11218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:43.253386 11217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:43.253499 11212 server_base.cc:1034] running on GCE node
W20260811 11:55:43.253399 11220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:43.254127 11212 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:43.254815 11212 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:43.255981 11212 hybrid_clock.cc:648] HybridClock initialized: now 1786449343255955 us; error 33 us; skew 500 ppm
Aug 11 11:55:43 dist-test-slave-zkpd krb5kdc[11095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449343, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:43.258613 11212 init.cc:377] Logged in from keytab as kudu/127.6.143.129@KRBTEST.COM (short username kudu)
I20260811 11:55:43.259318 11212 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:43.259405 11212 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:43.259627 11212 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:43.261667 11212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-0/data/instance:
uuid: "94df79b72dac414a972ee7c873b7bb79"
format_stamp: "Formatted at 2026-08-11 11:55:43 on dist-test-slave-zkpd"
server_key: "7ced44d6a27afa062fe77c8e207bd033"
server_key_iv: "ab6872283f9af13a5c9e1f5ffa783602"
server_key_version: "encryptionkey@0"
I20260811 11:55:43.262252 11212 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance:
uuid: "94df79b72dac414a972ee7c873b7bb79"
format_stamp: "Formatted at 2026-08-11 11:55:43 on dist-test-slave-zkpd"
server_key: "7ced44d6a27afa062fe77c8e207bd033"
server_key_iv: "ab6872283f9af13a5c9e1f5ffa783602"
server_key_version: "encryptionkey@0"
I20260811 11:55:43.265913 11212 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 11:55:43.268252 11226 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:43.269443 11212 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:55:43.269572 11212 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "94df79b72dac414a972ee7c873b7bb79"
format_stamp: "Formatted at 2026-08-11 11:55:43 on dist-test-slave-zkpd"
server_key: "7ced44d6a27afa062fe77c8e207bd033"
server_key_iv: "ab6872283f9af13a5c9e1f5ffa783602"
server_key_version: "encryptionkey@0"
I20260811 11:55:43.269682 11212 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:43.284940 11212 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:43.287876 11212 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:43.288080 11212 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:43.288628 11212 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:55:43.289422 11212 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:43.289499 11212 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:43.289565 11212 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:43.289601 11212 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:43.298870 11212 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:38327
I20260811 11:55:43.298890 11356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:38327 every 8 connection(s)
I20260811 11:55:43.299438 11212 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:55:43.304490 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 11212
I20260811 11:55:43.304613 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance
I20260811 11:55:43.304912 6718 external_mini_cluster.cc:1428] Setting key 56c76efc8850d02c05cd56a40a51fa19
Aug 11 11:55:43 dist-test-slave-zkpd krb5kdc[11095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449343, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:43.313844 11187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:43.301088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:41241 (local address 127.6.143.190:38681)
0811 11:55:43.301221 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:43.301225 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:43.302329 (+ 1104us) server_negotiation.cc:407] Connection header received
0811 11:55:43.303170 (+ 841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:43.303172 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:43.303218 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:43.303283 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:43.304521 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.305293 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.306538 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.306945 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.307070 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:43.309606 (+ 2536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:43.309611 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:43.309613 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:43.309641 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:43.311138 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:43.311853 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:43.311860 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:43.311863 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:43.311987 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:43.312486 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:43.312492 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:43.312493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:43.312824 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:43.312960 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:43.313437 (+ 477us) server_negotiation.cc:299] Negotiation successful
0811 11:55:43.313588 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 11:55:43.314414 11359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:43.301395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:38681 (local address 127.6.143.129:41241)
0811 11:55:43.302183 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:43.302230 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:43.302977 (+ 747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:43.303445 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:43.303451 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:43.303821 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:43.304379 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:43.304393 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.305418 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.305424 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:43.306422 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:43.306432 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.307083 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.307086 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:43.307224 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:43.307791 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:43.307807 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:43.309434 (+ 1627us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:43.311259 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:43.311268 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:43.311285 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:43.311709 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:43.312127 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:43.312133 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:43.312139 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:43.312345 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:43.312949 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:43.312954 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:43.313175 (+ 221us) client_negotiation.cc:769] Sending connection context
0811 11:55:43.313351 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 11:55:43.313609 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":111,"threads_started":1}
I20260811 11:55:43.315605 11357 heartbeater.cc:347] Connected to a master server at 127.6.143.190:38681
I20260811 11:55:43.315879 11357 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:43.316483 11357 heartbeater.cc:511] Master 127.6.143.190:38681 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.143.130@KRBTEST.COM; defaulting to no policy
I20260811 11:55:43.318097 11143 ts_manager.cc:194] Registered new tserver with Master: 94df79b72dac414a972ee7c873b7bb79 (127.6.143.129:38327)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.130@KRBTEST.COM" created.
I20260811 11:55:43.319418 11143 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.129@KRBTEST.COM'} at 127.6.143.129:41241
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
I20260811 11:55:43.332962 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:0
--local_ip_for_outbound_sockets=127.6.143.130
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:38681
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:39559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:43.445732 11362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:43.445986 11362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:43.446030 11362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:43.450201 11362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:43.450254 11362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:43.450314 11362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:55:43.455672 11362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:39559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:38681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:43.456665 11362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:43.457650 11362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:43.469132 11370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:43.469122 11367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:43.469123 11368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:43.469420 11362 server_base.cc:1034] running on GCE node
I20260811 11:55:43.469825 11362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:43.470446 11362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:43.471624 11362 hybrid_clock.cc:648] HybridClock initialized: now 1786449343471598 us; error 44 us; skew 500 ppm
Aug 11 11:55:43 dist-test-slave-zkpd krb5kdc[11095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449343, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:43.474417 11362 init.cc:377] Logged in from keytab as kudu/127.6.143.130@KRBTEST.COM (short username kudu)
I20260811 11:55:43.475168 11362 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:43.475261 11362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:43.475485 11362 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:43.477535 11362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-1/data/instance:
uuid: "9748bdab40884d4d96c138b0e51210fd"
format_stamp: "Formatted at 2026-08-11 11:55:43 on dist-test-slave-zkpd"
server_key: "13c747c5e11ceb65acfb73fc67c75c40"
server_key_iv: "c33077b8f176d3b46f1c80546b366dd7"
server_key_version: "encryptionkey@0"
I20260811 11:55:43.478098 11362 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance:
uuid: "9748bdab40884d4d96c138b0e51210fd"
format_stamp: "Formatted at 2026-08-11 11:55:43 on dist-test-slave-zkpd"
server_key: "13c747c5e11ceb65acfb73fc67c75c40"
server_key_iv: "c33077b8f176d3b46f1c80546b366dd7"
server_key_version: "encryptionkey@0"
I20260811 11:55:43.481803 11362 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260811 11:55:43.484133 11377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:43.485276 11362 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:43.485407 11362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "9748bdab40884d4d96c138b0e51210fd"
format_stamp: "Formatted at 2026-08-11 11:55:43 on dist-test-slave-zkpd"
server_key: "13c747c5e11ceb65acfb73fc67c75c40"
server_key_iv: "c33077b8f176d3b46f1c80546b366dd7"
server_key_version: "encryptionkey@0"
I20260811 11:55:43.485512 11362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:43.504473 11362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:43.507295 11362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:43.507504 11362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:43.508123 11362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:55:43.509006 11362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:43.509079 11362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:43.509157 11362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:43.509207 11362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:43.517788 11362 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:32925
I20260811 11:55:43.517810 11502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:32925 every 8 connection(s)
I20260811 11:55:43.518384 11362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
I20260811 11:55:43.519275 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 11362
I20260811 11:55:43.519414 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance
I20260811 11:55:43.519764 6718 external_mini_cluster.cc:1428] Setting key 39ed6defcb36c14f86d159d64ded766a
Aug 11 11:55:43 dist-test-slave-zkpd krb5kdc[11095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449343, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.143.131@KRBTEST.COM; defaulting to no policy
I20260811 11:55:43.531940 11187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:43.520227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:33943 (local address 127.6.143.190:38681)
0811 11:55:43.520393 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:43.520404 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:43.521721 (+ 1317us) server_negotiation.cc:407] Connection header received
0811 11:55:43.522547 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:43.522553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:43.522617 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:43.522710 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:43.523910 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.524433 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.525565 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.525788 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.525898 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:43.528349 (+ 2451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:43.528358 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:43.528361 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:43.528384 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:43.529758 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:43.530261 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:43.530265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:43.530267 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:43.530377 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:43.530716 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:43.530719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:43.530721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:43.530908 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:43.531017 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:43.531607 (+ 590us) server_negotiation.cc:299] Negotiation successful
0811 11:55:43.531730 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.131@KRBTEST.COM" created.
I20260811 11:55:43.532824 11505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:43.520668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:38681 (local address 127.6.143.130:33943)
0811 11:55:43.521581 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:43.521622 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:43.522351 (+ 729us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:43.522874 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:43.522882 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:43.523258 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:43.523737 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:43.523750 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.524568 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.524571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:43.525436 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:43.525446 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.525939 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.525944 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:43.526119 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:43.526699 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:43.526715 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:43.528187 (+ 1472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:43.529884 (+ 1697us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:43.529889 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:43.529904 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:43.530154 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:43.530475 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:43.530478 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:43.530480 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:43.530618 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:43.531016 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:43.531023 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:43.531364 (+ 341us) client_negotiation.cc:769] Sending connection context
0811 11:55:43.531546 (+ 182us) client_negotiation.cc:240] Negotiation successful
0811 11:55:43.531849 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":119,"threads_started":1}
I20260811 11:55:43.534035 11503 heartbeater.cc:347] Connected to a master server at 127.6.143.190:38681
I20260811 11:55:43.534291 11503 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:43.534821 11503 heartbeater.cc:511] Master 127.6.143.190:38681 requested a full tablet report, sending...
I20260811 11:55:43.535967 11143 ts_manager.cc:194] Registered new tserver with Master: 9748bdab40884d4d96c138b0e51210fd (127.6.143.130:32925)
I20260811 11:55:43.536859 11143 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.130@KRBTEST.COM'} at 127.6.143.130:33943
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
I20260811 11:55:43.548887 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:0
--local_ip_for_outbound_sockets=127.6.143.131
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:38681
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:39559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:43.661031 11508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:43.661296 11508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:43.661342 11508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:43.665450 11508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:43.665505 11508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:43.665604 11508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:55:43.671561 11508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:39559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:38681
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:43.672626 11508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:43.673647 11508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:43.683156 11513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:43.683183 11514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:43.683157 11516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:43.683353 11508 server_base.cc:1034] running on GCE node
I20260811 11:55:43.683887 11508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:43.684541 11508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:43.685720 11508 hybrid_clock.cc:648] HybridClock initialized: now 1786449343685703 us; error 26 us; skew 500 ppm
Aug 11 11:55:43 dist-test-slave-zkpd krb5kdc[11095](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449343, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:43.688464 11508 init.cc:377] Logged in from keytab as kudu/127.6.143.131@KRBTEST.COM (short username kudu)
I20260811 11:55:43.689121 11508 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:43.689179 11508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:43.689410 11508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:43.691309 11508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-2/data/instance:
uuid: "0841468236ce471e91b574ba38ef62cd"
format_stamp: "Formatted at 2026-08-11 11:55:43 on dist-test-slave-zkpd"
server_key: "1eecbd42232bf7785ddfc95fc558f004"
server_key_iv: "bd452f87d5fff92fe7dc896f248eb377"
server_key_version: "encryptionkey@0"
I20260811 11:55:43.691805 11508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance:
uuid: "0841468236ce471e91b574ba38ef62cd"
format_stamp: "Formatted at 2026-08-11 11:55:43 on dist-test-slave-zkpd"
server_key: "1eecbd42232bf7785ddfc95fc558f004"
server_key_iv: "bd452f87d5fff92fe7dc896f248eb377"
server_key_version: "encryptionkey@0"
I20260811 11:55:43.695461 11508 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 11:55:43.697872 11522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:43.699013 11508 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 11:55:43.699128 11508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "0841468236ce471e91b574ba38ef62cd"
format_stamp: "Formatted at 2026-08-11 11:55:43 on dist-test-slave-zkpd"
server_key: "1eecbd42232bf7785ddfc95fc558f004"
server_key_iv: "bd452f87d5fff92fe7dc896f248eb377"
server_key_version: "encryptionkey@0"
I20260811 11:55:43.699222 11508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:43.708923 11508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:43.711503 11508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:43.711683 11508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:43.712294 11508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:55:43.713095 11508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:43.713141 11508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:43.713203 11508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:43.713238 11508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:43.722114 11508 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:42857
I20260811 11:55:43.722131 11645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:42857 every 8 connection(s)
I20260811 11:55:43.722656 11508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:55:43.724977 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 11508
I20260811 11:55:43.725103 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance
I20260811 11:55:43.725488 6718 external_mini_cluster.cc:1428] Setting key 34c697680901dd5277f5e375ef72da2e
Aug 11 11:55:43 dist-test-slave-zkpd krb5kdc[11095](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449343, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:43.735433 11187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:43.724320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:50281 (local address 127.6.143.190:38681)
0811 11:55:43.724460 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:43.724463 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:43.725814 (+ 1351us) server_negotiation.cc:407] Connection header received
0811 11:55:43.726936 (+ 1122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:43.726940 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:43.726995 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:43.727080 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:43.728382 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.728904 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.729951 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.730157 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.730233 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:43.732473 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:43.732476 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:43.732478 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:43.732504 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:43.733701 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:43.734173 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:43.734175 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:43.734177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:43.734248 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:43.734520 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:43.734522 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:43.734523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:43.734682 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:43.734759 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:43.735232 (+ 473us) server_negotiation.cc:299] Negotiation successful
0811 11:55:43.735321 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 11:55:43.736275 11649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:43.724655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:38681 (local address 127.6.143.131:50281)
0811 11:55:43.725641 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:43.725697 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:43.726690 (+ 993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:43.727233 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:43.727240 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:43.727709 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:43.728254 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:43.728268 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.729009 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.729011 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:43.729847 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:43.729856 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.730277 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.730280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:43.730417 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:43.730925 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:43.730941 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:43.732329 (+ 1388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:43.733801 (+ 1472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:43.733806 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:43.733816 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:43.734090 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:43.734322 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:43.734324 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:43.734325 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:43.734447 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:43.734804 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:43.734809 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:43.735025 (+ 216us) client_negotiation.cc:769] Sending connection context
0811 11:55:43.735193 (+ 168us) client_negotiation.cc:240] Negotiation successful
0811 11:55:43.735429 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":124,"threads_started":1}
I20260811 11:55:43.737341 11646 heartbeater.cc:347] Connected to a master server at 127.6.143.190:38681
I20260811 11:55:43.737541 11646 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:43.737941 11646 heartbeater.cc:511] Master 127.6.143.190:38681 requested a full tablet report, sending...
I20260811 11:55:43.738972 11143 ts_manager.cc:194] Registered new tserver with Master: 0841468236ce471e91b574ba38ef62cd (127.6.143.131:42857)
I20260811 11:55:43.739537 11143 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.131@KRBTEST.COM'} at 127.6.143.131:50281
I20260811 11:55:43.750679 6718 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:55:43.759248 11187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:43.752517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51064 (local address 127.6.143.190:38681)
0811 11:55:43.752658 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:43.752711 (+ 53us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:43.752791 (+ 80us) server_negotiation.cc:407] Connection header received
0811 11:55:43.752888 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:43.752890 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:43.752927 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:43.752992 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:43.753610 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.754165 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.755049 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.755255 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.755332 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:43.756665 (+ 1333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:43.756668 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:43.756670 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:43.756691 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:43.757821 (+ 1130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:43.758180 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:43.758183 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:43.758184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:43.758250 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:43.758487 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:43.758489 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:43.758490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:43.758673 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:43.758770 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:43.758966 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 11:55:43.759050 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 11:55:43.761744 11143 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51064:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:55:43.764722 11143 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:55:43.779480 11662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:43.773798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:42857 (local address 127.0.0.1:55552)
0811 11:55:43.775023 (+ 1225us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:43.775033 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:43.775149 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:43.775688 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:43.775691 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:43.775713 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:43.775793 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:43.775798 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.777199 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.777203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:43.778138 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:43.778152 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.778943 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.778946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:43.779051 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:43.779088 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 11:55:43.779196 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 11:55:43.779315 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1172,"mutex_wait_us":3,"thread_start_us":84,"threads_started":1}
I20260811 11:55:43.779670 11664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:43.773980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55552 (local address 127.6.143.131:42857)
0811 11:55:43.774760 (+ 780us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:43.774772 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:43.775246 (+ 474us) server_negotiation.cc:407] Connection header received
0811 11:55:43.775324 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:43.775331 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:43.775531 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:43.775694 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:43.775937 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.777051 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.778268 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.778816 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.778942 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:43.779092 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:43.779431 (+ 339us) server_negotiation.cc:299] Negotiation successful
0811 11:55:43.779536 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":173,"threads_started":1}
I20260811 11:55:43.779807 11665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:43.774215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34676 (local address 127.6.143.130:32925)
0811 11:55:43.774699 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:43.774708 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:43.774745 (+ 37us) server_negotiation.cc:407] Connection header received
0811 11:55:43.774821 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:43.774826 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:43.775015 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:43.775164 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:43.775513 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.776728 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.778309 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.779095 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.779208 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:43.779326 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:43.779557 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 11:55:43.779664 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":167,"threads_started":1}
I20260811 11:55:43.780066 11661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:43.773191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42810 (local address 127.6.143.129:38327)
0811 11:55:43.773621 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:43.773630 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:43.774300 (+ 670us) server_negotiation.cc:407] Connection header received
0811 11:55:43.774392 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:43.774397 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:43.774639 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:43.774804 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:43.775128 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.776338 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.778085 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.778912 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.779022 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:43.779169 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:43.779775 (+ 606us) server_negotiation.cc:299] Negotiation successful
0811 11:55:43.779927 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":114,"threads_started":1}
I20260811 11:55:43.779663 11660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:43.773056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:38327 (local address 127.0.0.1:42810)
0811 11:55:43.773958 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:43.774005 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:43.774149 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:43.774810 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:43.774817 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:43.774842 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:43.774992 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:43.774999 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.776507 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.776511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:43.777954 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:43.777966 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.779238 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.779242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:43.779345 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:43.779377 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 11:55:43.779451 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 11:55:43.779522 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":732,"mutex_wait_us":26,"thread_start_us":67,"threads_started":1}
I20260811 11:55:43.780051 11663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:43.773233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:32925 (local address 127.0.0.1:34676)
0811 11:55:43.774410 (+ 1177us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:43.774423 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:43.774511 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:43.775299 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:43.775301 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:43.775316 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:43.775397 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:43.775402 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.776900 (+ 1498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.776904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:43.778165 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:43.778173 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.779204 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.779207 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:43.779299 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:43.779319 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 11:55:43.779401 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 11:55:43.779456 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1116,"mutex_wait_us":64,"thread_start_us":99,"threads_started":1}
I20260811 11:55:43.783209 11430 tablet_service.cc:1467] Processing CreateTablet for tablet ec569d9111d4434eb5ac9f68991c9c58 (DEFAULT_TABLE table=test-table [id=133ff2318ce6430a9c8285e59a603213]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:43.783299 11575 tablet_service.cc:1467] Processing CreateTablet for tablet ec569d9111d4434eb5ac9f68991c9c58 (DEFAULT_TABLE table=test-table [id=133ff2318ce6430a9c8285e59a603213]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:43.783299 11284 tablet_service.cc:1467] Processing CreateTablet for tablet ec569d9111d4434eb5ac9f68991c9c58 (DEFAULT_TABLE table=test-table [id=133ff2318ce6430a9c8285e59a603213]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:43.784279 11430 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec569d9111d4434eb5ac9f68991c9c58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:43.784359 11575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec569d9111d4434eb5ac9f68991c9c58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:43.784351 11284 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec569d9111d4434eb5ac9f68991c9c58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:43.790477 11666 tablet_bootstrap.cc:492] T ec569d9111d4434eb5ac9f68991c9c58 P 9748bdab40884d4d96c138b0e51210fd: Bootstrap starting.
I20260811 11:55:43.791788 11667 tablet_bootstrap.cc:492] T ec569d9111d4434eb5ac9f68991c9c58 P 94df79b72dac414a972ee7c873b7bb79: Bootstrap starting.
I20260811 11:55:43.791998 11668 tablet_bootstrap.cc:492] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd: Bootstrap starting.
I20260811 11:55:43.792985 11666 tablet_bootstrap.cc:654] T ec569d9111d4434eb5ac9f68991c9c58 P 9748bdab40884d4d96c138b0e51210fd: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:43.794106 11666 log.cc:824] T ec569d9111d4434eb5ac9f68991c9c58 P 9748bdab40884d4d96c138b0e51210fd: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:43.794190 11668 tablet_bootstrap.cc:654] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:43.794190 11667 tablet_bootstrap.cc:654] T ec569d9111d4434eb5ac9f68991c9c58 P 94df79b72dac414a972ee7c873b7bb79: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:43.795387 11667 log.cc:824] T ec569d9111d4434eb5ac9f68991c9c58 P 94df79b72dac414a972ee7c873b7bb79: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:43.795387 11668 log.cc:824] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:43.796149 11666 tablet_bootstrap.cc:492] T ec569d9111d4434eb5ac9f68991c9c58 P 9748bdab40884d4d96c138b0e51210fd: No bootstrap required, opened a new log
I20260811 11:55:43.796389 11666 ts_tablet_manager.cc:1397] T ec569d9111d4434eb5ac9f68991c9c58 P 9748bdab40884d4d96c138b0e51210fd: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 11:55:43.797269 11667 tablet_bootstrap.cc:492] T ec569d9111d4434eb5ac9f68991c9c58 P 94df79b72dac414a972ee7c873b7bb79: No bootstrap required, opened a new log
I20260811 11:55:43.797478 11667 ts_tablet_manager.cc:1397] T ec569d9111d4434eb5ac9f68991c9c58 P 94df79b72dac414a972ee7c873b7bb79: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 11:55:43.797570 11668 tablet_bootstrap.cc:492] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd: No bootstrap required, opened a new log
I20260811 11:55:43.797760 11668 ts_tablet_manager.cc:1397] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 11:55:43.800272 11667 raft_consensus.cc:348] T ec569d9111d4434eb5ac9f68991c9c58 P 94df79b72dac414a972ee7c873b7bb79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94df79b72dac414a972ee7c873b7bb79" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 38327 } } peers { permanent_uuid: "9748bdab40884d4d96c138b0e51210fd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 32925 } } peers { permanent_uuid: "0841468236ce471e91b574ba38ef62cd" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42857 } }
I20260811 11:55:43.800307 11666 raft_consensus.cc:348] T ec569d9111d4434eb5ac9f68991c9c58 P 9748bdab40884d4d96c138b0e51210fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94df79b72dac414a972ee7c873b7bb79" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 38327 } } peers { permanent_uuid: "9748bdab40884d4d96c138b0e51210fd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 32925 } } peers { permanent_uuid: "0841468236ce471e91b574ba38ef62cd" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42857 } }
I20260811 11:55:43.800565 11667 raft_consensus.cc:374] T ec569d9111d4434eb5ac9f68991c9c58 P 94df79b72dac414a972ee7c873b7bb79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:43.800623 11667 raft_consensus.cc:729] T ec569d9111d4434eb5ac9f68991c9c58 P 94df79b72dac414a972ee7c873b7bb79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 94df79b72dac414a972ee7c873b7bb79, State: Initialized, Role: FOLLOWER
I20260811 11:55:43.800653 11666 raft_consensus.cc:374] T ec569d9111d4434eb5ac9f68991c9c58 P 9748bdab40884d4d96c138b0e51210fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:43.800743 11666 raft_consensus.cc:729] T ec569d9111d4434eb5ac9f68991c9c58 P 9748bdab40884d4d96c138b0e51210fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9748bdab40884d4d96c138b0e51210fd, State: Initialized, Role: FOLLOWER
I20260811 11:55:43.801119 11667 consensus_queue.cc:260] T ec569d9111d4434eb5ac9f68991c9c58 P 94df79b72dac414a972ee7c873b7bb79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94df79b72dac414a972ee7c873b7bb79" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 38327 } } peers { permanent_uuid: "9748bdab40884d4d96c138b0e51210fd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 32925 } } peers { permanent_uuid: "0841468236ce471e91b574ba38ef62cd" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42857 } }
I20260811 11:55:43.801177 11666 consensus_queue.cc:260] T ec569d9111d4434eb5ac9f68991c9c58 P 9748bdab40884d4d96c138b0e51210fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94df79b72dac414a972ee7c873b7bb79" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 38327 } } peers { permanent_uuid: "9748bdab40884d4d96c138b0e51210fd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 32925 } } peers { permanent_uuid: "0841468236ce471e91b574ba38ef62cd" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42857 } }
I20260811 11:55:43.801779 11357 heartbeater.cc:503] Master 127.6.143.190:38681 was elected leader, sending a full tablet report...
I20260811 11:55:43.802126 11666 ts_tablet_manager.cc:1428] T ec569d9111d4434eb5ac9f68991c9c58 P 9748bdab40884d4d96c138b0e51210fd: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 11:55:43.802136 11667 ts_tablet_manager.cc:1428] T ec569d9111d4434eb5ac9f68991c9c58 P 94df79b72dac414a972ee7c873b7bb79: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 11:55:43.802471 11503 heartbeater.cc:503] Master 127.6.143.190:38681 was elected leader, sending a full tablet report...
I20260811 11:55:43.802392 11668 raft_consensus.cc:348] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94df79b72dac414a972ee7c873b7bb79" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 38327 } } peers { permanent_uuid: "9748bdab40884d4d96c138b0e51210fd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 32925 } } peers { permanent_uuid: "0841468236ce471e91b574ba38ef62cd" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42857 } }
I20260811 11:55:43.802639 11668 raft_consensus.cc:374] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:43.802688 11668 raft_consensus.cc:729] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0841468236ce471e91b574ba38ef62cd, State: Initialized, Role: FOLLOWER
I20260811 11:55:43.803115 11668 consensus_queue.cc:260] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94df79b72dac414a972ee7c873b7bb79" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 38327 } } peers { permanent_uuid: "9748bdab40884d4d96c138b0e51210fd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 32925 } } peers { permanent_uuid: "0841468236ce471e91b574ba38ef62cd" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42857 } }
I20260811 11:55:43.803802 11646 heartbeater.cc:503] Master 127.6.143.190:38681 was elected leader, sending a full tablet report...
I20260811 11:55:43.804051 11668 ts_tablet_manager.cc:1428] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 11:55:43.884934 11674 raft_consensus.cc:482] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:55:43.885160 11674 raft_consensus.cc:504] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94df79b72dac414a972ee7c873b7bb79" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 38327 } } peers { permanent_uuid: "9748bdab40884d4d96c138b0e51210fd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 32925 } } peers { permanent_uuid: "0841468236ce471e91b574ba38ef62cd" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42857 } }
I20260811 11:55:43.886289 11674 leader_election.cc:288] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 94df79b72dac414a972ee7c873b7bb79 (127.6.143.129:38327), 9748bdab40884d4d96c138b0e51210fd (127.6.143.130:32925)
I20260811 11:55:43.890744 11661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:43.886640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:57355 (local address 127.6.143.129:38327)
0811 11:55:43.886767 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:43.886772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:43.887135 (+ 363us) server_negotiation.cc:407] Connection header received
0811 11:55:43.887234 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:43.887236 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:43.887316 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:43.887399 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:43.887626 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.888331 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.889769 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.890169 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.890299 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:43.890355 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:43.890535 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 11:55:43.890606 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 11:55:43.890787 11665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:43.886652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:38643 (local address 127.6.143.130:32925)
0811 11:55:43.886767 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:43.886772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:43.886794 (+ 22us) server_negotiation.cc:407] Connection header received
0811 11:55:43.886858 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:43.886861 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:43.886903 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:43.886976 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:43.887323 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.888142 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.889722 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.890202 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.890334 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:43.890393 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:43.890589 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 11:55:43.890657 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 11:55:43.890787 11649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:43.886533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:32925 (local address 127.6.143.131:38643)
0811 11:55:43.886634 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:43.886651 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:43.886746 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:43.887017 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:43.887020 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:43.887043 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:43.887190 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:43.887200 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.888241 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.888244 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:43.889580 (+ 1336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:43.889592 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.890335 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.890340 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:43.890469 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:43.890488 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 11:55:43.890573 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 11:55:43.890646 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"mutex_wait_us":66}
I20260811 11:55:43.890799 11675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:43.886533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:38327 (local address 127.6.143.131:57355)
0811 11:55:43.887035 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:43.887048 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:43.887139 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:43.887422 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:43.887424 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:43.887439 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:43.887526 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:43.887530 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.888449 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.888452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:43.889665 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:43.889676 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.890300 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.890304 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:43.890404 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:43.890431 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 11:55:43.890516 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 11:55:43.890588 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"mutex_wait_us":142,"thread_start_us":134,"threads_started":1}
I20260811 11:55:43.891476 11452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec569d9111d4434eb5ac9f68991c9c58" candidate_uuid: "0841468236ce471e91b574ba38ef62cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9748bdab40884d4d96c138b0e51210fd" is_pre_election: true
I20260811 11:55:43.891664 11304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec569d9111d4434eb5ac9f68991c9c58" candidate_uuid: "0841468236ce471e91b574ba38ef62cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94df79b72dac414a972ee7c873b7bb79" is_pre_election: true
I20260811 11:55:43.891784 11452 raft_consensus.cc:2454] T ec569d9111d4434eb5ac9f68991c9c58 P 9748bdab40884d4d96c138b0e51210fd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0841468236ce471e91b574ba38ef62cd in term 0.
I20260811 11:55:43.891976 11304 raft_consensus.cc:2454] T ec569d9111d4434eb5ac9f68991c9c58 P 94df79b72dac414a972ee7c873b7bb79 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0841468236ce471e91b574ba38ef62cd in term 0.
I20260811 11:55:43.892325 11523 leader_election.cc:302] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0841468236ce471e91b574ba38ef62cd, 9748bdab40884d4d96c138b0e51210fd; no voters:
I20260811 11:55:43.892629 11674 raft_consensus.cc:2781] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:55:43.892716 11674 raft_consensus.cc:482] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:55:43.892760 11674 raft_consensus.cc:3037] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:43.893848 11674 raft_consensus.cc:504] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94df79b72dac414a972ee7c873b7bb79" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 38327 } } peers { permanent_uuid: "9748bdab40884d4d96c138b0e51210fd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 32925 } } peers { permanent_uuid: "0841468236ce471e91b574ba38ef62cd" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42857 } }
I20260811 11:55:43.894165 11674 leader_election.cc:288] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd [CANDIDATE]: Term 1 election: Requested vote from peers 94df79b72dac414a972ee7c873b7bb79 (127.6.143.129:38327), 9748bdab40884d4d96c138b0e51210fd (127.6.143.130:32925)
I20260811 11:55:43.894452 11304 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec569d9111d4434eb5ac9f68991c9c58" candidate_uuid: "0841468236ce471e91b574ba38ef62cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "94df79b72dac414a972ee7c873b7bb79"
I20260811 11:55:43.894511 11452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec569d9111d4434eb5ac9f68991c9c58" candidate_uuid: "0841468236ce471e91b574ba38ef62cd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9748bdab40884d4d96c138b0e51210fd"
I20260811 11:55:43.894577 11304 raft_consensus.cc:3037] T ec569d9111d4434eb5ac9f68991c9c58 P 94df79b72dac414a972ee7c873b7bb79 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:43.894651 11452 raft_consensus.cc:3037] T ec569d9111d4434eb5ac9f68991c9c58 P 9748bdab40884d4d96c138b0e51210fd [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:43.895939 11304 raft_consensus.cc:2454] T ec569d9111d4434eb5ac9f68991c9c58 P 94df79b72dac414a972ee7c873b7bb79 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0841468236ce471e91b574ba38ef62cd in term 1.
I20260811 11:55:43.896133 11452 raft_consensus.cc:2454] T ec569d9111d4434eb5ac9f68991c9c58 P 9748bdab40884d4d96c138b0e51210fd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0841468236ce471e91b574ba38ef62cd in term 1.
I20260811 11:55:43.896220 11524 leader_election.cc:302] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0841468236ce471e91b574ba38ef62cd, 94df79b72dac414a972ee7c873b7bb79; no voters:
I20260811 11:55:43.896375 11674 raft_consensus.cc:2781] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:43.896589 11674 raft_consensus.cc:686] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd [term 1 LEADER]: Becoming Leader. State: Replica: 0841468236ce471e91b574ba38ef62cd, State: Running, Role: LEADER
I20260811 11:55:43.896958 11674 consensus_queue.cc:237] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94df79b72dac414a972ee7c873b7bb79" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 38327 } } peers { permanent_uuid: "9748bdab40884d4d96c138b0e51210fd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 32925 } } peers { permanent_uuid: "0841468236ce471e91b574ba38ef62cd" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42857 } }
I20260811 11:55:43.899988 11141 catalog_manager.cc:5495] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd reported cstate change: term changed from 0 to 1, leader changed from <none> to 0841468236ce471e91b574ba38ef62cd (127.6.143.131). New cstate: current_term: 1 leader_uuid: "0841468236ce471e91b574ba38ef62cd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "94df79b72dac414a972ee7c873b7bb79" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 38327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9748bdab40884d4d96c138b0e51210fd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 32925 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0841468236ce471e91b574ba38ef62cd" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42857 } health_report { overall_health: HEALTHY } } }
I20260811 11:55:43.933363 11664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:43.929703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55558 (local address 127.6.143.131:42857)
0811 11:55:43.929840 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:43.929845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:43.929859 (+ 14us) server_negotiation.cc:407] Connection header received
0811 11:55:43.930014 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:43.930016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:43.930070 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:43.930185 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:55:43.930483 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.930993 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.931827 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:43.932083 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:43.932193 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:43.932528 (+ 335us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:55:43.932948 (+ 420us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:55:43.933080 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:43.933146 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 11:55:43.933203 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 11:55:43.939220 11452 raft_consensus.cc:1261] T ec569d9111d4434eb5ac9f68991c9c58 P 9748bdab40884d4d96c138b0e51210fd [term 1 FOLLOWER]: Refusing update from remote peer 0841468236ce471e91b574ba38ef62cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:43.939221 11304 raft_consensus.cc:1261] T ec569d9111d4434eb5ac9f68991c9c58 P 94df79b72dac414a972ee7c873b7bb79 [term 1 FOLLOWER]: Refusing update from remote peer 0841468236ce471e91b574ba38ef62cd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:43.940058 11676 consensus_queue.cc:1035] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "94df79b72dac414a972ee7c873b7bb79" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 38327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:43.940274 11674 consensus_queue.cc:1035] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd [LEADER]: Connected to new peer: Peer: permanent_uuid: "9748bdab40884d4d96c138b0e51210fd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 32925 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:43.949788 11682 mvcc.cc:205] Tried to move back new op lower bound from 7317296512767598592 to 7317296512604459008. Current Snapshot: MvccSnapshot[applied={T|T < 7317296512767598592}]
I20260811 11:55:43.958640 11143 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:55:43.958808 11143 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51064:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:55:43.961020 11143 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8549bae94c4b480a815785e214edf37b: Sending DeleteTablet for 3 replicas of tablet ec569d9111d4434eb5ac9f68991c9c58
I20260811 11:55:43.961855 11575 tablet_service.cc:1514] Processing DeleteTablet for tablet ec569d9111d4434eb5ac9f68991c9c58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:43 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:55552
I20260811 11:55:43.961846 11284 tablet_service.cc:1514] Processing DeleteTablet for tablet ec569d9111d4434eb5ac9f68991c9c58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:43 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:42810
I20260811 11:55:43.961884 11430 tablet_service.cc:1514] Processing DeleteTablet for tablet ec569d9111d4434eb5ac9f68991c9c58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:43 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:34676
I20260811 11:55:43.962381 11693 tablet_replica.cc:330] T ec569d9111d4434eb5ac9f68991c9c58 P 94df79b72dac414a972ee7c873b7bb79: stopping tablet replica
I20260811 11:55:43.962584 11692 tablet_replica.cc:330] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd: stopping tablet replica
I20260811 11:55:43.962697 11693 raft_consensus.cc:2229] T ec569d9111d4434eb5ac9f68991c9c58 P 94df79b72dac414a972ee7c873b7bb79 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:55:43.962841 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 11212
I20260811 11:55:43.963020 11693 raft_consensus.cc:2258] T ec569d9111d4434eb5ac9f68991c9c58 P 94df79b72dac414a972ee7c873b7bb79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:43.963050 11692 raft_consensus.cc:2229] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:55:43.963581 11692 raft_consensus.cc:2258] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:43.965106 11694 tablet_replica.cc:330] T ec569d9111d4434eb5ac9f68991c9c58 P 9748bdab40884d4d96c138b0e51210fd: stopping tablet replica
I20260811 11:55:43.965469 11694 raft_consensus.cc:2229] T ec569d9111d4434eb5ac9f68991c9c58 P 9748bdab40884d4d96c138b0e51210fd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:55:43.965795 11694 raft_consensus.cc:2258] T ec569d9111d4434eb5ac9f68991c9c58 P 9748bdab40884d4d96c138b0e51210fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:43.967463 11692 ts_tablet_manager.cc:1905] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:43.967480 11694 ts_tablet_manager.cc:1905] T ec569d9111d4434eb5ac9f68991c9c58 P 9748bdab40884d4d96c138b0e51210fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:43.970199 11692 ts_tablet_manager.cc:1918] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:55:43.970296 11692 log.cc:1196] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-2/wal/wals/ec569d9111d4434eb5ac9f68991c9c58
I20260811 11:55:43.970633 11692 ts_tablet_manager.cc:1939] T ec569d9111d4434eb5ac9f68991c9c58 P 0841468236ce471e91b574ba38ef62cd: Deleting consensus metadata
I20260811 11:55:43.971088 11694 ts_tablet_manager.cc:1918] T ec569d9111d4434eb5ac9f68991c9c58 P 9748bdab40884d4d96c138b0e51210fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:55:43.971256 11694 log.cc:1196] T ec569d9111d4434eb5ac9f68991c9c58 P 9748bdab40884d4d96c138b0e51210fd: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786449319379472-6718-0/minicluster-data/ts-1/wal/wals/ec569d9111d4434eb5ac9f68991c9c58
I20260811 11:55:43.971637 11694 ts_tablet_manager.cc:1939] T ec569d9111d4434eb5ac9f68991c9c58 P 9748bdab40884d4d96c138b0e51210fd: Deleting consensus metadata
I20260811 11:55:43.971732 11125 catalog_manager.cc:4841] TS 0841468236ce471e91b574ba38ef62cd (127.6.143.131:42857): tablet ec569d9111d4434eb5ac9f68991c9c58 (table test-table [id=133ff2318ce6430a9c8285e59a603213]) successfully deleted
I20260811 11:55:43.972911 11125 catalog_manager.cc:4841] TS 9748bdab40884d4d96c138b0e51210fd (127.6.143.130:32925): tablet ec569d9111d4434eb5ac9f68991c9c58 (table test-table [id=133ff2318ce6430a9c8285e59a603213]) successfully deleted
W20260811 11:55:43.973932 11126 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.143.129:38327 (error 108)
I20260811 11:55:43.974031 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 11362
I20260811 11:55:43.980396 11663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:43.976246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:38327 (local address 127.0.0.1:42814)
0811 11:55:43.980197 (+ 3951us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:43.980280 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.129:38327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3839}
W20260811 11:55:43.980581 11126 catalog_manager.cc:4584] TS 94df79b72dac414a972ee7c873b7bb79 (127.6.143.129:38327): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ec569d9111d4434eb5ac9f68991c9c58: Network error: Client connection negotiation failed: client connection to 127.6.143.129:38327: connect: Connection refused (error 111)
I20260811 11:55:43.985605 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 11508
I20260811 11:55:43.992519 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 11110
2026-08-11T11:55:43Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2346 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T11:55:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:55:44Z Disabled control of system clock
I20260811 11:55:44.026108 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:37267
--webserver_interface=127.6.143.190
--webserver_port=0
--builtin_ntp_servers=127.6.143.148:41081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:37267
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 11:55:44.146212 11699 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:44.146477 11699 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:44.146517 11699 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:44.150535 11699 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:55:44.150584 11699 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:44.150599 11699 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:44.150615 11699 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:55:44.150629 11699 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:55:44.155278 11699 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:41081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:37267
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:37267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11699
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:44.156322 11699 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:44.157661 11699 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:44.166150 11707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:44.166143 11710 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:44.166143 11706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:44.166468 11699 server_base.cc:1034] running on GCE node
I20260811 11:55:44.167119 11699 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:44.168520 11699 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:44.169734 11699 hybrid_clock.cc:648] HybridClock initialized: now 1786449344169699 us; error 51 us; skew 500 ppm
I20260811 11:55:44.171765 11699 webserver.cc:460] Webserver started at http://127.6.143.190:45543/ using document root <none> and password file <none>
I20260811 11:55:44.172441 11699 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:44.172537 11699 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:44.172776 11699 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:44.174849 11699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "a94e42e3f86f4bd5840d24eb87d8989e"
format_stamp: "Formatted at 2026-08-11 11:55:44 on dist-test-slave-zkpd"
server_key: "08f0a240e7e2b80f2ab731c0aba15419"
server_key_iv: "22f9bbb94851b78e4b1e1a167d24eae7"
server_key_version: "encryptionkey@0"
I20260811 11:55:44.175431 11699 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "a94e42e3f86f4bd5840d24eb87d8989e"
format_stamp: "Formatted at 2026-08-11 11:55:44 on dist-test-slave-zkpd"
server_key: "08f0a240e7e2b80f2ab731c0aba15419"
server_key_iv: "22f9bbb94851b78e4b1e1a167d24eae7"
server_key_version: "encryptionkey@0"
I20260811 11:55:44.179041 11699 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 11:55:44.181473 11716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:44.182665 11699 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 11:55:44.182808 11699 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "a94e42e3f86f4bd5840d24eb87d8989e"
format_stamp: "Formatted at 2026-08-11 11:55:44 on dist-test-slave-zkpd"
server_key: "08f0a240e7e2b80f2ab731c0aba15419"
server_key_iv: "22f9bbb94851b78e4b1e1a167d24eae7"
server_key_version: "encryptionkey@0"
I20260811 11:55:44.182924 11699 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:44.188150 11699 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:44.188750 11699 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:44.188941 11699 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:44.195853 11699 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:37267
I20260811 11:55:44.195868 11775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:37267 every 8 connection(s)
I20260811 11:55:44.196827 11699 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:55:44.200054 11776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:44.202281 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 11699
I20260811 11:55:44.202422 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:55:44.202689 6718 external_mini_cluster.cc:1428] Setting key 22da886acdc89225009d1bea818b7e33
I20260811 11:55:44.205847 11776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a94e42e3f86f4bd5840d24eb87d8989e: Bootstrap starting.
I20260811 11:55:44.208088 11779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:44.204042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51890 (local address 127.6.143.190:37267)
0811 11:55:44.204647 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:44.204662 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:44.204696 (+ 34us) server_negotiation.cc:407] Connection header received
0811 11:55:44.205314 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:44.205329 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:44.205677 (+ 348us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:44.205974 (+ 297us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:44.206775 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:44.206780 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:44.206792 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:55:44.206802 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:44.206857 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:55:44.206872 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:44.206982 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:44.207102 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 11:55:44.207345 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":135,"threads_started":1}
I20260811 11:55:44.208485 11776 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a94e42e3f86f4bd5840d24eb87d8989e: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:44.209270 11776 log.cc:824] T 00000000000000000000000000000000 P a94e42e3f86f4bd5840d24eb87d8989e: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:44.211179 11776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a94e42e3f86f4bd5840d24eb87d8989e: No bootstrap required, opened a new log
I20260811 11:55:44.214931 11776 raft_consensus.cc:348] T 00000000000000000000000000000000 P a94e42e3f86f4bd5840d24eb87d8989e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94e42e3f86f4bd5840d24eb87d8989e" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 37267 } }
I20260811 11:55:44.215144 11776 raft_consensus.cc:374] T 00000000000000000000000000000000 P a94e42e3f86f4bd5840d24eb87d8989e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:44.215183 11776 raft_consensus.cc:729] T 00000000000000000000000000000000 P a94e42e3f86f4bd5840d24eb87d8989e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a94e42e3f86f4bd5840d24eb87d8989e, State: Initialized, Role: FOLLOWER
I20260811 11:55:44.215538 11776 consensus_queue.cc:260] T 00000000000000000000000000000000 P a94e42e3f86f4bd5840d24eb87d8989e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94e42e3f86f4bd5840d24eb87d8989e" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 37267 } }
I20260811 11:55:44.215647 11776 raft_consensus.cc:388] T 00000000000000000000000000000000 P a94e42e3f86f4bd5840d24eb87d8989e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:55:44.215690 11776 raft_consensus.cc:482] T 00000000000000000000000000000000 P a94e42e3f86f4bd5840d24eb87d8989e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:55:44.215758 11776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a94e42e3f86f4bd5840d24eb87d8989e [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:44.216668 11776 raft_consensus.cc:504] T 00000000000000000000000000000000 P a94e42e3f86f4bd5840d24eb87d8989e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94e42e3f86f4bd5840d24eb87d8989e" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 37267 } }
I20260811 11:55:44.216964 11776 leader_election.cc:302] T 00000000000000000000000000000000 P a94e42e3f86f4bd5840d24eb87d8989e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a94e42e3f86f4bd5840d24eb87d8989e; no voters:
I20260811 11:55:44.217206 11776 leader_election.cc:288] T 00000000000000000000000000000000 P a94e42e3f86f4bd5840d24eb87d8989e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:55:44.217320 11781 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a94e42e3f86f4bd5840d24eb87d8989e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:44.217556 11781 raft_consensus.cc:686] T 00000000000000000000000000000000 P a94e42e3f86f4bd5840d24eb87d8989e [term 1 LEADER]: Becoming Leader. State: Replica: a94e42e3f86f4bd5840d24eb87d8989e, State: Running, Role: LEADER
I20260811 11:55:44.217866 11781 consensus_queue.cc:237] T 00000000000000000000000000000000 P a94e42e3f86f4bd5840d24eb87d8989e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94e42e3f86f4bd5840d24eb87d8989e" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 37267 } }
I20260811 11:55:44.218537 11776 sys_catalog.cc:564] T 00000000000000000000000000000000 P a94e42e3f86f4bd5840d24eb87d8989e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:55:44.219460 11785 sys_catalog.cc:455] T 00000000000000000000000000000000 P a94e42e3f86f4bd5840d24eb87d8989e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a94e42e3f86f4bd5840d24eb87d8989e. Latest consensus state: current_term: 1 leader_uuid: "a94e42e3f86f4bd5840d24eb87d8989e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94e42e3f86f4bd5840d24eb87d8989e" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 37267 } } }
I20260811 11:55:44.219609 11785 sys_catalog.cc:458] T 00000000000000000000000000000000 P a94e42e3f86f4bd5840d24eb87d8989e [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:44.220002 11784 sys_catalog.cc:455] T 00000000000000000000000000000000 P a94e42e3f86f4bd5840d24eb87d8989e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a94e42e3f86f4bd5840d24eb87d8989e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a94e42e3f86f4bd5840d24eb87d8989e" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 37267 } } }
I20260811 11:55:44.220067 11792 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:55:44.220099 11784 sys_catalog.cc:458] T 00000000000000000000000000000000 P a94e42e3f86f4bd5840d24eb87d8989e [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:44.223336 11792 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:55:44.228892 11792 catalog_manager.cc:1347] Generated new cluster ID: b916eaa535584ea4b25ea835a61f8ccb
I20260811 11:55:44.228966 11792 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:55:44.236908 11792 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:55:44.237210 11792 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:55:44.241345 11792 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a94e42e3f86f4bd5840d24eb87d8989e: Generated new TSK 0
I20260811 11:55:44.242055 11792 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:55:44.249853 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:0
--local_ip_for_outbound_sockets=127.6.143.129
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:37267
--builtin_ntp_servers=127.6.143.148:41081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 11:55:44.364127 11802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:44.364393 11802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:44.364470 11802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:44.368408 11802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:44.368481 11802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:44.368568 11802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:55:44.373220 11802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:41081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:37267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:44.374243 11802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:44.375591 11802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:44.385106 11807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:44.385118 11802 server_base.cc:1034] running on GCE node
W20260811 11:55:44.385078 11808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:44.385059 11810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:44.385715 11802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:44.386368 11802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:44.387531 11802 hybrid_clock.cc:648] HybridClock initialized: now 1786449344387508 us; error 34 us; skew 500 ppm
I20260811 11:55:44.389349 11802 webserver.cc:460] Webserver started at http://127.6.143.129:41503/ using document root <none> and password file <none>
I20260811 11:55:44.389940 11802 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:44.390003 11802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:44.390239 11802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:44.392297 11802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-0/data/instance:
uuid: "3c083582fab045b1b2bfa7700c32c5de"
format_stamp: "Formatted at 2026-08-11 11:55:44 on dist-test-slave-zkpd"
server_key: "a42bb8ed84e17d02b41b29917148f357"
server_key_iv: "1e0fd6f48692f98b009f9a6577594aa8"
server_key_version: "encryptionkey@0"
I20260811 11:55:44.392846 11802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance:
uuid: "3c083582fab045b1b2bfa7700c32c5de"
format_stamp: "Formatted at 2026-08-11 11:55:44 on dist-test-slave-zkpd"
server_key: "a42bb8ed84e17d02b41b29917148f357"
server_key_iv: "1e0fd6f48692f98b009f9a6577594aa8"
server_key_version: "encryptionkey@0"
I20260811 11:55:44.396486 11802 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 11:55:44.398752 11816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:44.399986 11802 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260811 11:55:44.400137 11802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "3c083582fab045b1b2bfa7700c32c5de"
format_stamp: "Formatted at 2026-08-11 11:55:44 on dist-test-slave-zkpd"
server_key: "a42bb8ed84e17d02b41b29917148f357"
server_key_iv: "1e0fd6f48692f98b009f9a6577594aa8"
server_key_version: "encryptionkey@0"
I20260811 11:55:44.400234 11802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:44.404598 11802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:44.405160 11802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:44.405337 11802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:44.405867 11802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:55:44.406751 11802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:44.406804 11802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:44.406868 11802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:44.406899 11802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:44.415405 11802 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:35447
I20260811 11:55:44.415428 11942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:35447 every 8 connection(s)
I20260811 11:55:44.416344 11802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:55:44.416728 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 11802
I20260811 11:55:44.416857 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance
I20260811 11:55:44.417179 6718 external_mini_cluster.cc:1428] Setting key 8e0192c7aecb57289e3103bb5b62d97d
I20260811 11:55:44.419526 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:0
--local_ip_for_outbound_sockets=127.6.143.130
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:37267
--builtin_ntp_servers=127.6.143.148:41081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 11:55:44.424877 11779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:44.418360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:40769 (local address 127.6.143.190:37267)
0811 11:55:44.418504 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:44.418508 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:44.421726 (+ 3218us) server_negotiation.cc:407] Connection header received
0811 11:55:44.423089 (+ 1363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:44.423093 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:44.423134 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:44.423241 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:44.424394 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:44.424397 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:44.424400 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:55:44.424412 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:44.424436 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:55:44.424451 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:44.424545 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:44.424704 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 11:55:44.424748 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 11:55:44.425693 11946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:44.418722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:37267 (local address 127.6.143.129:40769)
0811 11:55:44.421591 (+ 2869us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:44.421630 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:44.422877 (+ 1247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:44.423409 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:44.423416 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:44.423503 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:44.424139 (+ 636us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:55:44.424141 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:44.424181 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:55:44.424249 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:44.424543 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:44.424554 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:44.424562 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 11:55:44.424735 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 11:55:44.424972 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2471,"thread_start_us":145,"threads_started":1}
I20260811 11:55:44.426894 11943 heartbeater.cc:347] Connected to a master server at 127.6.143.190:37267
I20260811 11:55:44.427160 11943 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:44.427668 11943 heartbeater.cc:511] Master 127.6.143.190:37267 requested a full tablet report, sending...
I20260811 11:55:44.429528 11734 ts_manager.cc:194] Registered new tserver with Master: 3c083582fab045b1b2bfa7700c32c5de (127.6.143.129:35447)
W20260811 11:55:44.534788 11947 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:44.535015 11947 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:44.535051 11947 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:44.538990 11947 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:44.539039 11947 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:44.539093 11947 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:55:44.543550 11947 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:41081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:37267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11947
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:44.544543 11947 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:44.545841 11947 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:44.555307 11956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:44.555316 11952 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:44.555320 11954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:44.555362 11947 server_base.cc:1034] running on GCE node
I20260811 11:55:44.556006 11947 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:44.556628 11947 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:44.557814 11947 hybrid_clock.cc:648] HybridClock initialized: now 1786449344557798 us; error 31 us; skew 500 ppm
I20260811 11:55:44.559650 11947 webserver.cc:460] Webserver started at http://127.6.143.130:40963/ using document root <none> and password file <none>
I20260811 11:55:44.560295 11947 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:44.560395 11947 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:44.560626 11947 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:44.562644 11947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-1/data/instance:
uuid: "2a060f30eb2a47e2beca447295625c88"
format_stamp: "Formatted at 2026-08-11 11:55:44 on dist-test-slave-zkpd"
server_key: "9f93d1164c777312dd5745310b9fcdaa"
server_key_iv: "449bcbdd99d329fc57dd4f372321a561"
server_key_version: "encryptionkey@0"
I20260811 11:55:44.563210 11947 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance:
uuid: "2a060f30eb2a47e2beca447295625c88"
format_stamp: "Formatted at 2026-08-11 11:55:44 on dist-test-slave-zkpd"
server_key: "9f93d1164c777312dd5745310b9fcdaa"
server_key_iv: "449bcbdd99d329fc57dd4f372321a561"
server_key_version: "encryptionkey@0"
I20260811 11:55:44.566823 11947 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 11:55:44.569187 11962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:44.570322 11947 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 11:55:44.570456 11947 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "2a060f30eb2a47e2beca447295625c88"
format_stamp: "Formatted at 2026-08-11 11:55:44 on dist-test-slave-zkpd"
server_key: "9f93d1164c777312dd5745310b9fcdaa"
server_key_iv: "449bcbdd99d329fc57dd4f372321a561"
server_key_version: "encryptionkey@0"
I20260811 11:55:44.570561 11947 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:44.575052 11947 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:44.575605 11947 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:44.575786 11947 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:44.576346 11947 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:55:44.577229 11947 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:44.577298 11947 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:44.577364 11947 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:44.577399 11947 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:44.586014 11947 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:38339
I20260811 11:55:44.586078 12084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:38339 every 8 connection(s)
I20260811 11:55:44.586918 11947 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
I20260811 11:55:44.592886 11779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:44.588774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:52223 (local address 127.6.143.190:37267)
0811 11:55:44.588929 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:44.588934 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:44.590100 (+ 1166us) server_negotiation.cc:407] Connection header received
0811 11:55:44.591315 (+ 1215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:44.591317 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:44.591359 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:44.591433 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:44.592495 (+ 1062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:44.592498 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:44.592501 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:55:44.592512 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:44.592531 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:55:44.592540 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:44.592616 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:44.592743 (+ 127us) server_negotiation.cc:299] Negotiation successful
0811 11:55:44.592780 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 11:55:44.593667 12088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:44.589235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:37267 (local address 127.6.143.130:52223)
0811 11:55:44.589944 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:44.589983 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:44.591128 (+ 1145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:44.591590 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:44.591597 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:44.591679 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:44.592234 (+ 555us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:55:44.592236 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:44.592274 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:55:44.592340 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:44.592616 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:44.592622 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:44.592626 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:55:44.592756 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 11:55:44.592939 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":132,"threads_started":1}
I20260811 11:55:44.594828 12086 heartbeater.cc:347] Connected to a master server at 127.6.143.190:37267
I20260811 11:55:44.595057 12086 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:44.595454 12086 heartbeater.cc:511] Master 127.6.143.190:37267 requested a full tablet report, sending...
I20260811 11:55:44.596284 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 11947
I20260811 11:55:44.596451 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance
I20260811 11:55:44.596539 11734 ts_manager.cc:194] Registered new tserver with Master: 2a060f30eb2a47e2beca447295625c88 (127.6.143.130:38339)
I20260811 11:55:44.596783 6718 external_mini_cluster.cc:1428] Setting key b5b9fb3c665d5938f77d6f1b21b5e780
I20260811 11:55:44.598667 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:0
--local_ip_for_outbound_sockets=127.6.143.131
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:37267
--builtin_ntp_servers=127.6.143.148:41081
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 11:55:44.714504 12089 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:44.714790 12089 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:44.714880 12089 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:44.719410 12089 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:44.719491 12089 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:44.719586 12089 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:55:44.724781 12089 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:41081
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:37267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12089
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:44.725873 12089 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:44.727247 12089 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:44.736901 12095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:44.736979 12089 server_base.cc:1034] running on GCE node
W20260811 11:55:44.736872 12096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:44.736882 12098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:44.737490 12089 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:44.738101 12089 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:44.739265 12089 hybrid_clock.cc:648] HybridClock initialized: now 1786449344739245 us; error 30 us; skew 500 ppm
I20260811 11:55:44.740989 12089 webserver.cc:460] Webserver started at http://127.6.143.131:34595/ using document root <none> and password file <none>
I20260811 11:55:44.741745 12089 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:44.741906 12089 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:44.742101 12089 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:44.744563 12089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-2/data/instance:
uuid: "e3deb86bcdfa4f808090aff2f4e95f69"
format_stamp: "Formatted at 2026-08-11 11:55:44 on dist-test-slave-zkpd"
server_key: "ecf4a10815c9cb942dcf8d3fb210e907"
server_key_iv: "dcd3a6b4e9d23eed1a407e3dd4076442"
server_key_version: "encryptionkey@0"
I20260811 11:55:44.745085 12089 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance:
uuid: "e3deb86bcdfa4f808090aff2f4e95f69"
format_stamp: "Formatted at 2026-08-11 11:55:44 on dist-test-slave-zkpd"
server_key: "ecf4a10815c9cb942dcf8d3fb210e907"
server_key_iv: "dcd3a6b4e9d23eed1a407e3dd4076442"
server_key_version: "encryptionkey@0"
I20260811 11:55:44.748693 12089 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 11:55:44.751278 12104 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:44.752626 12089 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 11:55:44.752724 12089 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "e3deb86bcdfa4f808090aff2f4e95f69"
format_stamp: "Formatted at 2026-08-11 11:55:44 on dist-test-slave-zkpd"
server_key: "ecf4a10815c9cb942dcf8d3fb210e907"
server_key_iv: "dcd3a6b4e9d23eed1a407e3dd4076442"
server_key_version: "encryptionkey@0"
I20260811 11:55:44.752799 12089 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:44.757129 12089 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:44.757645 12089 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:44.757777 12089 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:44.758249 12089 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:55:44.759083 12089 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:44.759130 12089 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:44.759167 12089 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:44.759189 12089 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:44.767854 12089 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:37457
I20260811 11:55:44.767882 12232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:37457 every 8 connection(s)
I20260811 11:55:44.768728 12089 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:55:44.774684 11779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:44.770495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:47819 (local address 127.6.143.190:37267)
0811 11:55:44.770624 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:44.770627 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:44.771678 (+ 1051us) server_negotiation.cc:407] Connection header received
0811 11:55:44.772983 (+ 1305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:44.772986 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:44.773023 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:44.773099 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:44.774228 (+ 1129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:44.774231 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:44.774234 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:55:44.774246 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:44.774286 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:55:44.774295 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:44.774365 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:44.774531 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 11:55:44.774583 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 11:55:44.775393 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 12089
I20260811 11:55:44.775514 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance
I20260811 11:55:44.775646 12235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:44.770877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:37267 (local address 127.6.143.131:47819)
0811 11:55:44.771521 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:44.771560 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:44.772793 (+ 1233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:44.773254 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:44.773260 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:44.773343 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:44.773974 (+ 631us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:55:44.773976 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:44.774014 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:55:44.774080 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:44.774403 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:44.774410 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:44.774415 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:55:44.774550 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 11:55:44.774810 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":114,"threads_started":1}
I20260811 11:55:44.775832 6718 external_mini_cluster.cc:1428] Setting key c6de8b223fe3e1be07e5a715983ac32d
I20260811 11:55:44.776942 12233 heartbeater.cc:347] Connected to a master server at 127.6.143.190:37267
I20260811 11:55:44.777192 12233 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:44.777595 12233 heartbeater.cc:511] Master 127.6.143.190:37267 requested a full tablet report, sending...
I20260811 11:55:44.778601 11734 ts_manager.cc:194] Registered new tserver with Master: e3deb86bcdfa4f808090aff2f4e95f69 (127.6.143.131:37457)
I20260811 11:55:44.789510 6718 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:55:44.792907 11779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:44.791336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51896 (local address 127.6.143.190:37267)
0811 11:55:44.791481 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:44.791486 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:44.791636 (+ 150us) server_negotiation.cc:407] Connection header received
0811 11:55:44.791732 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:44.791734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:44.791765 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:44.791829 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:44.792562 (+ 733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:44.792564 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:44.792566 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:55:44.792575 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:44.792590 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:55:44.792597 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:44.792663 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:44.792704 (+ 41us) server_negotiation.cc:299] Negotiation successful
0811 11:55:44.792736 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 11:55:44.795809 11734 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:51896:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:55:44.798894 11734 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:55:44.810642 12245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:44.807090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:37457 (local address 127.0.0.1:47604)
0811 11:55:44.807886 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:44.807959 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:44.808076 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:44.808927 (+ 851us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:44.808935 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:44.809016 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:44.810011 (+ 995us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:55:44.810012 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:44.810020 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:55:44.810036 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:44.810373 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:44.810376 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:44.810380 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:55:44.810465 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 11:55:44.810500 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":62,"threads_started":1}
I20260811 11:55:44.810650 12249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:44.808115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49946 (local address 127.6.143.129:35447)
0811 11:55:44.808572 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:44.808584 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:44.808816 (+ 232us) server_negotiation.cc:407] Connection header received
0811 11:55:44.808883 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:44.808891 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:44.809061 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:44.809202 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:44.810036 (+ 834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:44.810043 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:44.810056 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:55:44.810069 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:44.810115 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:55:44.810148 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:44.810255 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:44.810395 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 11:55:44.810475 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":143,"threads_started":1}
I20260811 11:55:44.810714 12246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:44.807206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47604 (local address 127.6.143.131:37457)
0811 11:55:44.807606 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:44.807617 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:44.808185 (+ 568us) server_negotiation.cc:407] Connection header received
0811 11:55:44.808248 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:44.808257 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:44.808470 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:44.808613 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:44.810156 (+ 1543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:44.810162 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:44.810174 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:55:44.810185 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:44.810228 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:55:44.810262 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:44.810366 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:44.810477 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 11:55:44.810545 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":170,"threads_started":1}
I20260811 11:55:44.811239 12247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:44.807168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:38339 (local address 127.0.0.1:57044)
0811 11:55:44.808432 (+ 1264us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:44.808446 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:44.808530 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:44.809287 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:44.809293 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:44.809312 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:44.809750 (+ 438us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:55:44.809774 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:44.809799 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:55:44.809871 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:44.811010 (+ 1139us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:44.811013 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:44.811017 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:55:44.811097 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 11:55:44.811130 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1200,"thread_start_us":101,"threads_started":1}
I20260811 11:55:44.811579 12244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:44.807691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:35447 (local address 127.0.0.1:49946)
0811 11:55:44.808709 (+ 1018us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:44.808721 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:44.808803 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:44.809542 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:44.809546 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:44.809559 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:44.809872 (+ 313us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:55:44.809873 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:44.809882 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:55:44.809910 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:44.810237 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:44.810247 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:44.810254 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 11:55:44.810366 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 11:55:44.810448 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":957,"thread_start_us":95,"threads_started":1}
I20260811 11:55:44.811869 12248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:44.807848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57044 (local address 127.6.143.130:38339)
0811 11:55:44.808292 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:44.808305 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:44.808635 (+ 330us) server_negotiation.cc:407] Connection header received
0811 11:55:44.808695 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:44.808700 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:44.808895 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:44.809029 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:44.810036 (+ 1007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:44.810043 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:44.810056 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:55:44.810069 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:44.810115 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:55:44.810148 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:44.810255 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:44.811199 (+ 944us) server_negotiation.cc:299] Negotiation successful
0811 11:55:44.811270 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":125,"threads_started":1}
I20260811 11:55:44.813884 12158 tablet_service.cc:1467] Processing CreateTablet for tablet 50039408eb5f4e9ba76853cfc016c1a4 (DEFAULT_TABLE table=test-table [id=96812315a05c44c7ac39219ef60d335d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:44.814147 11875 tablet_service.cc:1467] Processing CreateTablet for tablet 50039408eb5f4e9ba76853cfc016c1a4 (DEFAULT_TABLE table=test-table [id=96812315a05c44c7ac39219ef60d335d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:44.814508 12017 tablet_service.cc:1467] Processing CreateTablet for tablet 50039408eb5f4e9ba76853cfc016c1a4 (DEFAULT_TABLE table=test-table [id=96812315a05c44c7ac39219ef60d335d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:44.815054 12158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50039408eb5f4e9ba76853cfc016c1a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:44.815217 11875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50039408eb5f4e9ba76853cfc016c1a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:44.815390 12017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 50039408eb5f4e9ba76853cfc016c1a4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:44.822238 12250 tablet_bootstrap.cc:492] T 50039408eb5f4e9ba76853cfc016c1a4 P 3c083582fab045b1b2bfa7700c32c5de: Bootstrap starting.
I20260811 11:55:44.823522 12251 tablet_bootstrap.cc:492] T 50039408eb5f4e9ba76853cfc016c1a4 P 2a060f30eb2a47e2beca447295625c88: Bootstrap starting.
I20260811 11:55:44.824236 12252 tablet_bootstrap.cc:492] T 50039408eb5f4e9ba76853cfc016c1a4 P e3deb86bcdfa4f808090aff2f4e95f69: Bootstrap starting.
I20260811 11:55:44.824708 12250 tablet_bootstrap.cc:654] T 50039408eb5f4e9ba76853cfc016c1a4 P 3c083582fab045b1b2bfa7700c32c5de: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:44.825603 12250 log.cc:824] T 50039408eb5f4e9ba76853cfc016c1a4 P 3c083582fab045b1b2bfa7700c32c5de: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:44.825676 12251 tablet_bootstrap.cc:654] T 50039408eb5f4e9ba76853cfc016c1a4 P 2a060f30eb2a47e2beca447295625c88: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:44.826304 12252 tablet_bootstrap.cc:654] T 50039408eb5f4e9ba76853cfc016c1a4 P e3deb86bcdfa4f808090aff2f4e95f69: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:44.826467 12251 log.cc:824] T 50039408eb5f4e9ba76853cfc016c1a4 P 2a060f30eb2a47e2beca447295625c88: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:44.827068 12252 log.cc:824] T 50039408eb5f4e9ba76853cfc016c1a4 P e3deb86bcdfa4f808090aff2f4e95f69: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:44.827692 12250 tablet_bootstrap.cc:492] T 50039408eb5f4e9ba76853cfc016c1a4 P 3c083582fab045b1b2bfa7700c32c5de: No bootstrap required, opened a new log
I20260811 11:55:44.827943 12250 ts_tablet_manager.cc:1397] T 50039408eb5f4e9ba76853cfc016c1a4 P 3c083582fab045b1b2bfa7700c32c5de: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 11:55:44.828437 12251 tablet_bootstrap.cc:492] T 50039408eb5f4e9ba76853cfc016c1a4 P 2a060f30eb2a47e2beca447295625c88: No bootstrap required, opened a new log
I20260811 11:55:44.828624 12251 ts_tablet_manager.cc:1397] T 50039408eb5f4e9ba76853cfc016c1a4 P 2a060f30eb2a47e2beca447295625c88: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 11:55:44.828979 12252 tablet_bootstrap.cc:492] T 50039408eb5f4e9ba76853cfc016c1a4 P e3deb86bcdfa4f808090aff2f4e95f69: No bootstrap required, opened a new log
I20260811 11:55:44.829157 12252 ts_tablet_manager.cc:1397] T 50039408eb5f4e9ba76853cfc016c1a4 P e3deb86bcdfa4f808090aff2f4e95f69: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 11:55:44.831830 12250 raft_consensus.cc:348] T 50039408eb5f4e9ba76853cfc016c1a4 P 3c083582fab045b1b2bfa7700c32c5de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3deb86bcdfa4f808090aff2f4e95f69" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 37457 } } peers { permanent_uuid: "2a060f30eb2a47e2beca447295625c88" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 38339 } } peers { permanent_uuid: "3c083582fab045b1b2bfa7700c32c5de" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 35447 } }
I20260811 11:55:44.832129 12250 raft_consensus.cc:374] T 50039408eb5f4e9ba76853cfc016c1a4 P 3c083582fab045b1b2bfa7700c32c5de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:44.832195 12250 raft_consensus.cc:729] T 50039408eb5f4e9ba76853cfc016c1a4 P 3c083582fab045b1b2bfa7700c32c5de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3c083582fab045b1b2bfa7700c32c5de, State: Initialized, Role: FOLLOWER
I20260811 11:55:44.832670 12250 consensus_queue.cc:260] T 50039408eb5f4e9ba76853cfc016c1a4 P 3c083582fab045b1b2bfa7700c32c5de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3deb86bcdfa4f808090aff2f4e95f69" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 37457 } } peers { permanent_uuid: "2a060f30eb2a47e2beca447295625c88" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 38339 } } peers { permanent_uuid: "3c083582fab045b1b2bfa7700c32c5de" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 35447 } }
I20260811 11:55:44.833269 11943 heartbeater.cc:503] Master 127.6.143.190:37267 was elected leader, sending a full tablet report...
I20260811 11:55:44.833156 12251 raft_consensus.cc:348] T 50039408eb5f4e9ba76853cfc016c1a4 P 2a060f30eb2a47e2beca447295625c88 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3deb86bcdfa4f808090aff2f4e95f69" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 37457 } } peers { permanent_uuid: "2a060f30eb2a47e2beca447295625c88" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 38339 } } peers { permanent_uuid: "3c083582fab045b1b2bfa7700c32c5de" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 35447 } }
I20260811 11:55:44.833389 12251 raft_consensus.cc:374] T 50039408eb5f4e9ba76853cfc016c1a4 P 2a060f30eb2a47e2beca447295625c88 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:44.833451 12251 raft_consensus.cc:729] T 50039408eb5f4e9ba76853cfc016c1a4 P 2a060f30eb2a47e2beca447295625c88 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2a060f30eb2a47e2beca447295625c88, State: Initialized, Role: FOLLOWER
I20260811 11:55:44.833634 12250 ts_tablet_manager.cc:1428] T 50039408eb5f4e9ba76853cfc016c1a4 P 3c083582fab045b1b2bfa7700c32c5de: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 11:55:44.833503 12252 raft_consensus.cc:348] T 50039408eb5f4e9ba76853cfc016c1a4 P e3deb86bcdfa4f808090aff2f4e95f69 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3deb86bcdfa4f808090aff2f4e95f69" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 37457 } } peers { permanent_uuid: "2a060f30eb2a47e2beca447295625c88" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 38339 } } peers { permanent_uuid: "3c083582fab045b1b2bfa7700c32c5de" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 35447 } }
I20260811 11:55:44.833745 12252 raft_consensus.cc:374] T 50039408eb5f4e9ba76853cfc016c1a4 P e3deb86bcdfa4f808090aff2f4e95f69 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:44.833793 12252 raft_consensus.cc:729] T 50039408eb5f4e9ba76853cfc016c1a4 P e3deb86bcdfa4f808090aff2f4e95f69 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e3deb86bcdfa4f808090aff2f4e95f69, State: Initialized, Role: FOLLOWER
I20260811 11:55:44.833914 12251 consensus_queue.cc:260] T 50039408eb5f4e9ba76853cfc016c1a4 P 2a060f30eb2a47e2beca447295625c88 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3deb86bcdfa4f808090aff2f4e95f69" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 37457 } } peers { permanent_uuid: "2a060f30eb2a47e2beca447295625c88" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 38339 } } peers { permanent_uuid: "3c083582fab045b1b2bfa7700c32c5de" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 35447 } }
I20260811 11:55:44.834172 12252 consensus_queue.cc:260] T 50039408eb5f4e9ba76853cfc016c1a4 P e3deb86bcdfa4f808090aff2f4e95f69 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3deb86bcdfa4f808090aff2f4e95f69" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 37457 } } peers { permanent_uuid: "2a060f30eb2a47e2beca447295625c88" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 38339 } } peers { permanent_uuid: "3c083582fab045b1b2bfa7700c32c5de" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 35447 } }
I20260811 11:55:44.834681 12086 heartbeater.cc:503] Master 127.6.143.190:37267 was elected leader, sending a full tablet report...
I20260811 11:55:44.835064 12252 ts_tablet_manager.cc:1428] T 50039408eb5f4e9ba76853cfc016c1a4 P e3deb86bcdfa4f808090aff2f4e95f69: Time spent starting tablet: real 0.006s user 0.000s sys 0.007s
I20260811 11:55:44.835386 12233 heartbeater.cc:503] Master 127.6.143.190:37267 was elected leader, sending a full tablet report...
I20260811 11:55:44.836019 12251 ts_tablet_manager.cc:1428] T 50039408eb5f4e9ba76853cfc016c1a4 P 2a060f30eb2a47e2beca447295625c88: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260811 11:55:44.838167 12087 tablet.cc:2369] T 50039408eb5f4e9ba76853cfc016c1a4 P 2a060f30eb2a47e2beca447295625c88: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:55:44.912534 12257 raft_consensus.cc:482] T 50039408eb5f4e9ba76853cfc016c1a4 P 3c083582fab045b1b2bfa7700c32c5de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:55:44.912760 12257 raft_consensus.cc:504] T 50039408eb5f4e9ba76853cfc016c1a4 P 3c083582fab045b1b2bfa7700c32c5de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3deb86bcdfa4f808090aff2f4e95f69" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 37457 } } peers { permanent_uuid: "2a060f30eb2a47e2beca447295625c88" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 38339 } } peers { permanent_uuid: "3c083582fab045b1b2bfa7700c32c5de" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 35447 } }
I20260811 11:55:44.913870 12257 leader_election.cc:288] T 50039408eb5f4e9ba76853cfc016c1a4 P 3c083582fab045b1b2bfa7700c32c5de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e3deb86bcdfa4f808090aff2f4e95f69 (127.6.143.131:37457), 2a060f30eb2a47e2beca447295625c88 (127.6.143.130:38339)
I20260811 11:55:44.916023 12248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:44.914246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:52921 (local address 127.6.143.130:38339)
0811 11:55:44.914384 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:44.914389 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:44.914407 (+ 18us) server_negotiation.cc:407] Connection header received
0811 11:55:44.914460 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:44.914464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:44.914506 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:44.914596 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:44.915384 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:44.915388 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:44.915391 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:55:44.915416 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:44.915436 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:55:44.915447 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:44.915549 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:44.915822 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 11:55:44.915864 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 11:55:44.916044 11946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:44.914105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:38339 (local address 127.6.143.129:52921)
0811 11:55:44.914253 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:44.914270 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:44.914373 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:44.914616 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:44.914619 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:44.914635 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:44.915169 (+ 534us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:55:44.915173 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:44.915187 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:55:44.915219 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:44.915727 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:44.915730 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:44.915732 (+ 2us) client_negotiation.cc:769] Sending connection context
0811 11:55:44.915821 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 11:55:44.915875 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 11:55:44.916074 12260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:44.914110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:37457 (local address 127.6.143.129:59889)
0811 11:55:44.914562 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:44.914574 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:44.914681 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:44.914940 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:44.914943 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:44.914958 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:44.915405 (+ 447us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:55:44.915408 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:44.915419 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:55:44.915447 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:44.915722 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:44.915725 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:44.915728 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 11:55:44.915823 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 11:55:44.915884 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":151,"thread_start_us":77,"threads_started":1}
I20260811 11:55:44.916893 12246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:44.914250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:59889 (local address 127.6.143.131:37457)
0811 11:55:44.914393 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:44.914397 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:44.914648 (+ 251us) server_negotiation.cc:407] Connection header received
0811 11:55:44.914798 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:44.914800 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:44.914833 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:44.914916 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:44.915577 (+ 661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:44.915581 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:44.915583 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:55:44.915593 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:44.915612 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:55:44.915619 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:44.915689 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:44.915845 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 11:55:44.916734 (+ 889us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 11:55:44.916847 12037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50039408eb5f4e9ba76853cfc016c1a4" candidate_uuid: "3c083582fab045b1b2bfa7700c32c5de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a060f30eb2a47e2beca447295625c88" is_pre_election: true
I20260811 11:55:44.917171 12037 raft_consensus.cc:2454] T 50039408eb5f4e9ba76853cfc016c1a4 P 2a060f30eb2a47e2beca447295625c88 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c083582fab045b1b2bfa7700c32c5de in term 0.
I20260811 11:55:44.917518 12180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50039408eb5f4e9ba76853cfc016c1a4" candidate_uuid: "3c083582fab045b1b2bfa7700c32c5de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3deb86bcdfa4f808090aff2f4e95f69" is_pre_election: true
W20260811 11:55:44.917732 11945 tablet.cc:2369] T 50039408eb5f4e9ba76853cfc016c1a4 P 3c083582fab045b1b2bfa7700c32c5de: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:55:44.917704 11817 leader_election.cc:302] T 50039408eb5f4e9ba76853cfc016c1a4 P 3c083582fab045b1b2bfa7700c32c5de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2a060f30eb2a47e2beca447295625c88, 3c083582fab045b1b2bfa7700c32c5de; no voters:
I20260811 11:55:44.917850 12180 raft_consensus.cc:2454] T 50039408eb5f4e9ba76853cfc016c1a4 P e3deb86bcdfa4f808090aff2f4e95f69 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3c083582fab045b1b2bfa7700c32c5de in term 0.
I20260811 11:55:44.918236 12257 raft_consensus.cc:2781] T 50039408eb5f4e9ba76853cfc016c1a4 P 3c083582fab045b1b2bfa7700c32c5de [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:55:44.918326 12257 raft_consensus.cc:482] T 50039408eb5f4e9ba76853cfc016c1a4 P 3c083582fab045b1b2bfa7700c32c5de [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:55:44.918375 12257 raft_consensus.cc:3037] T 50039408eb5f4e9ba76853cfc016c1a4 P 3c083582fab045b1b2bfa7700c32c5de [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:44.919485 12257 raft_consensus.cc:504] T 50039408eb5f4e9ba76853cfc016c1a4 P 3c083582fab045b1b2bfa7700c32c5de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3deb86bcdfa4f808090aff2f4e95f69" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 37457 } } peers { permanent_uuid: "2a060f30eb2a47e2beca447295625c88" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 38339 } } peers { permanent_uuid: "3c083582fab045b1b2bfa7700c32c5de" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 35447 } }
I20260811 11:55:44.919819 12257 leader_election.cc:288] T 50039408eb5f4e9ba76853cfc016c1a4 P 3c083582fab045b1b2bfa7700c32c5de [CANDIDATE]: Term 1 election: Requested vote from peers e3deb86bcdfa4f808090aff2f4e95f69 (127.6.143.131:37457), 2a060f30eb2a47e2beca447295625c88 (127.6.143.130:38339)
I20260811 11:55:44.920172 12180 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50039408eb5f4e9ba76853cfc016c1a4" candidate_uuid: "3c083582fab045b1b2bfa7700c32c5de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e3deb86bcdfa4f808090aff2f4e95f69"
I20260811 11:55:44.920325 12180 raft_consensus.cc:3037] T 50039408eb5f4e9ba76853cfc016c1a4 P e3deb86bcdfa4f808090aff2f4e95f69 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:44.921351 12180 raft_consensus.cc:2454] T 50039408eb5f4e9ba76853cfc016c1a4 P e3deb86bcdfa4f808090aff2f4e95f69 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c083582fab045b1b2bfa7700c32c5de in term 1.
I20260811 11:55:44.921594 12037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "50039408eb5f4e9ba76853cfc016c1a4" candidate_uuid: "3c083582fab045b1b2bfa7700c32c5de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2a060f30eb2a47e2beca447295625c88"
I20260811 11:55:44.921649 11819 leader_election.cc:302] T 50039408eb5f4e9ba76853cfc016c1a4 P 3c083582fab045b1b2bfa7700c32c5de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3c083582fab045b1b2bfa7700c32c5de, e3deb86bcdfa4f808090aff2f4e95f69; no voters:
I20260811 11:55:44.921751 12037 raft_consensus.cc:3037] T 50039408eb5f4e9ba76853cfc016c1a4 P 2a060f30eb2a47e2beca447295625c88 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:44.921829 12257 raft_consensus.cc:2781] T 50039408eb5f4e9ba76853cfc016c1a4 P 3c083582fab045b1b2bfa7700c32c5de [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:44.922041 12257 raft_consensus.cc:686] T 50039408eb5f4e9ba76853cfc016c1a4 P 3c083582fab045b1b2bfa7700c32c5de [term 1 LEADER]: Becoming Leader. State: Replica: 3c083582fab045b1b2bfa7700c32c5de, State: Running, Role: LEADER
I20260811 11:55:44.922392 12257 consensus_queue.cc:237] T 50039408eb5f4e9ba76853cfc016c1a4 P 3c083582fab045b1b2bfa7700c32c5de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3deb86bcdfa4f808090aff2f4e95f69" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 37457 } } peers { permanent_uuid: "2a060f30eb2a47e2beca447295625c88" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 38339 } } peers { permanent_uuid: "3c083582fab045b1b2bfa7700c32c5de" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 35447 } }
I20260811 11:55:44.923048 12037 raft_consensus.cc:2454] T 50039408eb5f4e9ba76853cfc016c1a4 P 2a060f30eb2a47e2beca447295625c88 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3c083582fab045b1b2bfa7700c32c5de in term 1.
I20260811 11:55:44.925848 11732 catalog_manager.cc:5495] T 50039408eb5f4e9ba76853cfc016c1a4 P 3c083582fab045b1b2bfa7700c32c5de reported cstate change: term changed from 0 to 1, leader changed from <none> to 3c083582fab045b1b2bfa7700c32c5de (127.6.143.129). New cstate: current_term: 1 leader_uuid: "3c083582fab045b1b2bfa7700c32c5de" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e3deb86bcdfa4f808090aff2f4e95f69" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 37457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2a060f30eb2a47e2beca447295625c88" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 38339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3c083582fab045b1b2bfa7700c32c5de" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 35447 } health_report { overall_health: HEALTHY } } }
I20260811 11:55:44.934090 12249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:44.932474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49958 (local address 127.6.143.129:35447)
0811 11:55:44.932587 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:44.932592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:44.932647 (+ 55us) server_negotiation.cc:407] Connection header received
0811 11:55:44.932871 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:44.932874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:44.932926 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:44.933016 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:44.933710 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:44.933713 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:44.933716 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:55:44.933727 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:44.933746 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:55:44.933755 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:44.933860 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:44.933937 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 11:55:44.933973 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 11:55:44.939388 12180 raft_consensus.cc:1261] T 50039408eb5f4e9ba76853cfc016c1a4 P e3deb86bcdfa4f808090aff2f4e95f69 [term 1 FOLLOWER]: Refusing update from remote peer 3c083582fab045b1b2bfa7700c32c5de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:44.939414 12037 raft_consensus.cc:1261] T 50039408eb5f4e9ba76853cfc016c1a4 P 2a060f30eb2a47e2beca447295625c88 [term 1 FOLLOWER]: Refusing update from remote peer 3c083582fab045b1b2bfa7700c32c5de: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:44.940130 12261 consensus_queue.cc:1035] T 50039408eb5f4e9ba76853cfc016c1a4 P 3c083582fab045b1b2bfa7700c32c5de [LEADER]: Connected to new peer: Peer: permanent_uuid: "e3deb86bcdfa4f808090aff2f4e95f69" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 37457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:44.940315 12257 consensus_queue.cc:1035] T 50039408eb5f4e9ba76853cfc016c1a4 P 3c083582fab045b1b2bfa7700c32c5de [LEADER]: Connected to new peer: Peer: permanent_uuid: "2a060f30eb2a47e2beca447295625c88" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 38339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:44.956501 11732 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:51896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:55:44.956668 11732 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:51896:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:55:44.958895 11732 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a94e42e3f86f4bd5840d24eb87d8989e: Sending DeleteTablet for 3 replicas of tablet 50039408eb5f4e9ba76853cfc016c1a4
I20260811 11:55:44.959586 12158 tablet_service.cc:1514] Processing DeleteTablet for tablet 50039408eb5f4e9ba76853cfc016c1a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:44 UTC) from {username='slave'} at 127.0.0.1:47604
I20260811 11:55:44.959651 12017 tablet_service.cc:1514] Processing DeleteTablet for tablet 50039408eb5f4e9ba76853cfc016c1a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:44 UTC) from {username='slave'} at 127.0.0.1:57044
I20260811 11:55:44.959787 11875 tablet_service.cc:1514] Processing DeleteTablet for tablet 50039408eb5f4e9ba76853cfc016c1a4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:44 UTC) from {username='slave'} at 127.0.0.1:49946
I20260811 11:55:44.960062 12275 tablet_replica.cc:330] T 50039408eb5f4e9ba76853cfc016c1a4 P e3deb86bcdfa4f808090aff2f4e95f69: stopping tablet replica
I20260811 11:55:44.960309 12276 tablet_replica.cc:330] T 50039408eb5f4e9ba76853cfc016c1a4 P 3c083582fab045b1b2bfa7700c32c5de: stopping tablet replica
I20260811 11:55:44.960368 12275 raft_consensus.cc:2229] T 50039408eb5f4e9ba76853cfc016c1a4 P e3deb86bcdfa4f808090aff2f4e95f69 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:55:44.960690 12275 raft_consensus.cc:2258] T 50039408eb5f4e9ba76853cfc016c1a4 P e3deb86bcdfa4f808090aff2f4e95f69 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:44.960772 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 11802
I20260811 11:55:44.960708 12276 raft_consensus.cc:2229] T 50039408eb5f4e9ba76853cfc016c1a4 P 3c083582fab045b1b2bfa7700c32c5de [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:55:44.963368 12277 tablet_replica.cc:330] T 50039408eb5f4e9ba76853cfc016c1a4 P 2a060f30eb2a47e2beca447295625c88: stopping tablet replica
I20260811 11:55:44.963651 12277 raft_consensus.cc:2229] T 50039408eb5f4e9ba76853cfc016c1a4 P 2a060f30eb2a47e2beca447295625c88 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:55:44.963945 12277 raft_consensus.cc:2258] T 50039408eb5f4e9ba76853cfc016c1a4 P 2a060f30eb2a47e2beca447295625c88 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:44.966009 12277 ts_tablet_manager.cc:1905] T 50039408eb5f4e9ba76853cfc016c1a4 P 2a060f30eb2a47e2beca447295625c88: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:44.966833 12275 ts_tablet_manager.cc:1905] T 50039408eb5f4e9ba76853cfc016c1a4 P e3deb86bcdfa4f808090aff2f4e95f69: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:44.968971 12275 ts_tablet_manager.cc:1918] T 50039408eb5f4e9ba76853cfc016c1a4 P e3deb86bcdfa4f808090aff2f4e95f69: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:55:44.969085 12275 log.cc:1196] T 50039408eb5f4e9ba76853cfc016c1a4 P e3deb86bcdfa4f808090aff2f4e95f69: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-2/wal/wals/50039408eb5f4e9ba76853cfc016c1a4
I20260811 11:55:44.969372 12275 ts_tablet_manager.cc:1939] T 50039408eb5f4e9ba76853cfc016c1a4 P e3deb86bcdfa4f808090aff2f4e95f69: Deleting consensus metadata
I20260811 11:55:44.969532 12277 ts_tablet_manager.cc:1918] T 50039408eb5f4e9ba76853cfc016c1a4 P 2a060f30eb2a47e2beca447295625c88: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:55:44.969636 12277 log.cc:1196] T 50039408eb5f4e9ba76853cfc016c1a4 P 2a060f30eb2a47e2beca447295625c88: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786449319379472-6718-0/minicluster-data/ts-1/wal/wals/50039408eb5f4e9ba76853cfc016c1a4
I20260811 11:55:44.969950 12277 ts_tablet_manager.cc:1939] T 50039408eb5f4e9ba76853cfc016c1a4 P 2a060f30eb2a47e2beca447295625c88: Deleting consensus metadata
I20260811 11:55:44.970548 11719 catalog_manager.cc:4841] TS e3deb86bcdfa4f808090aff2f4e95f69 (127.6.143.131:37457): tablet 50039408eb5f4e9ba76853cfc016c1a4 (table test-table [id=96812315a05c44c7ac39219ef60d335d]) successfully deleted
I20260811 11:55:44.971113 11717 catalog_manager.cc:4841] TS 2a060f30eb2a47e2beca447295625c88 (127.6.143.130:38339): tablet 50039408eb5f4e9ba76853cfc016c1a4 (table test-table [id=96812315a05c44c7ac39219ef60d335d]) successfully deleted
W20260811 11:55:44.975073 11720 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.143.129:35447 (error 108)
I20260811 11:55:44.975790 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 11947
I20260811 11:55:44.976141 12245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:44.975770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:35447 (local address 127.0.0.1:49968)
0811 11:55:44.975971 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:44.976048 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.129:35447: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 11:55:44.977366 11720 catalog_manager.cc:4584] TS 3c083582fab045b1b2bfa7700c32c5de (127.6.143.129:35447): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 50039408eb5f4e9ba76853cfc016c1a4: Network error: Client connection negotiation failed: client connection to 127.6.143.129:35447: connect: Connection refused (error 111)
I20260811 11:55:44.983484 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 12089
I20260811 11:55:44.990593 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 11699
2026-08-11T11:55:44Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (995 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 11:55:45.013211 6718 mini_oidc.cc:152] Starting JWKS server
I20260811 11:55:45.014802 6718 webserver.cc:460] Webserver started at https://127.0.0.1:44901/ using document root <none> and password file <none>
I20260811 11:55:45.015004 6718 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 11:55:45.015269 6718 webserver.cc:460] Webserver started at http://127.0.0.1:36815/ using document root <none> and password file <none>
2026-08-11T11:55:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:55:45Z Disabled control of system clock
I20260811 11:55:45.024966 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:46791
--webserver_interface=127.6.143.190
--webserver_port=0
--builtin_ntp_servers=127.6.143.148:41319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:46791
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44901/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449319379472-6718-0/testchainca.cert with env {}
W20260811 11:55:45.137681 12285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:45.138002 12285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:45.138078 12285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:45.142086 12285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:55:45.142194 12285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:45.142234 12285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:45.142261 12285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:55:45.142316 12285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 11:55:45.142349 12285 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 11:55:45.142382 12285 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44901/jwks.json
I20260811 11:55:45.147758 12285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:41319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:46791
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:46791
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44901/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449319379472-6718-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:45.149313 12285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:45.150341 12285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:45.158975 12293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:45.159020 12290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:45.159020 12291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:45.159192 12285 server_base.cc:1034] running on GCE node
I20260811 11:55:45.159777 12285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:45.160988 12285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:45.162185 12285 hybrid_clock.cc:648] HybridClock initialized: now 1786449345162170 us; error 28 us; skew 500 ppm
I20260811 11:55:45.164150 12285 webserver.cc:460] Webserver started at http://127.6.143.190:42963/ using document root <none> and password file <none>
I20260811 11:55:45.164736 12285 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:45.164830 12285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:45.165069 12285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:45.167109 12285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "451334be41054bb498861a2fadb117a5"
format_stamp: "Formatted at 2026-08-11 11:55:45 on dist-test-slave-zkpd"
server_key: "dc8fc454c73f25701f22024502ffd19f"
server_key_iv: "120b5de499c5ac5cf532b5e8b4aea9de"
server_key_version: "encryptionkey@0"
I20260811 11:55:45.167688 12285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "451334be41054bb498861a2fadb117a5"
format_stamp: "Formatted at 2026-08-11 11:55:45 on dist-test-slave-zkpd"
server_key: "dc8fc454c73f25701f22024502ffd19f"
server_key_iv: "120b5de499c5ac5cf532b5e8b4aea9de"
server_key_version: "encryptionkey@0"
I20260811 11:55:45.171416 12285 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 11:55:45.173816 12299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:45.174973 12285 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:45.175114 12285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "451334be41054bb498861a2fadb117a5"
format_stamp: "Formatted at 2026-08-11 11:55:45 on dist-test-slave-zkpd"
server_key: "dc8fc454c73f25701f22024502ffd19f"
server_key_iv: "120b5de499c5ac5cf532b5e8b4aea9de"
server_key_version: "encryptionkey@0"
I20260811 11:55:45.175220 12285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:45.190589 12285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:45.191215 12285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:45.191404 12285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:45.198347 12285 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:46791
I20260811 11:55:45.198354 12356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:46791 every 8 connection(s)
I20260811 11:55:45.199235 12285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:55:45.200724 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 12285
I20260811 11:55:45.200863 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:55:45.201130 6718 external_mini_cluster.cc:1428] Setting key f6a5ee7eed150f5a3508286f28d5fbb5
I20260811 11:55:45.202399 12357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:45.209270 12357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 451334be41054bb498861a2fadb117a5: Bootstrap starting.
I20260811 11:55:45.211395 12360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:45.202510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41372 (local address 127.6.143.190:46791)
0811 11:55:45.203203 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:45.203224 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:45.203266 (+ 42us) server_negotiation.cc:407] Connection header received
0811 11:55:45.204140 (+ 874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:45.204163 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:45.204640 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:45.205019 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:45.205447 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:45.206865 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:45.207698 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:45.208427 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:45.208587 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:45.209415 (+ 828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:45.209422 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:45.209436 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:55:45.209911 (+ 475us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:45.209977 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:55:45.210004 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:45.210100 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:45.210237 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 11:55:45.210518 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":130,"threads_started":1}
I20260811 11:55:45.211966 12357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 451334be41054bb498861a2fadb117a5: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:45.212782 12357 log.cc:824] T 00000000000000000000000000000000 P 451334be41054bb498861a2fadb117a5: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:45.214327 6718 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:55:45.214880 12357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 451334be41054bb498861a2fadb117a5: No bootstrap required, opened a new log
I20260811 11:55:45.218405 12357 raft_consensus.cc:348] T 00000000000000000000000000000000 P 451334be41054bb498861a2fadb117a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "451334be41054bb498861a2fadb117a5" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46791 } }
I20260811 11:55:45.218639 12357 raft_consensus.cc:374] T 00000000000000000000000000000000 P 451334be41054bb498861a2fadb117a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:45.218721 12357 raft_consensus.cc:729] T 00000000000000000000000000000000 P 451334be41054bb498861a2fadb117a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 451334be41054bb498861a2fadb117a5, State: Initialized, Role: FOLLOWER
I20260811 11:55:45.219146 12357 consensus_queue.cc:260] T 00000000000000000000000000000000 P 451334be41054bb498861a2fadb117a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "451334be41054bb498861a2fadb117a5" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46791 } }
I20260811 11:55:45.219295 12357 raft_consensus.cc:388] T 00000000000000000000000000000000 P 451334be41054bb498861a2fadb117a5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:55:45.219380 12357 raft_consensus.cc:482] T 00000000000000000000000000000000 P 451334be41054bb498861a2fadb117a5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:55:45.219501 12357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 451334be41054bb498861a2fadb117a5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:45.220508 12357 raft_consensus.cc:504] T 00000000000000000000000000000000 P 451334be41054bb498861a2fadb117a5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "451334be41054bb498861a2fadb117a5" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46791 } }
I20260811 11:55:45.220796 12357 leader_election.cc:302] T 00000000000000000000000000000000 P 451334be41054bb498861a2fadb117a5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 451334be41054bb498861a2fadb117a5; no voters:
I20260811 11:55:45.221060 12357 leader_election.cc:288] T 00000000000000000000000000000000 P 451334be41054bb498861a2fadb117a5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:55:45.221176 12363 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 451334be41054bb498861a2fadb117a5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:45.221422 12363 raft_consensus.cc:686] T 00000000000000000000000000000000 P 451334be41054bb498861a2fadb117a5 [term 1 LEADER]: Becoming Leader. State: Replica: 451334be41054bb498861a2fadb117a5, State: Running, Role: LEADER
I20260811 11:55:45.221731 12363 consensus_queue.cc:237] T 00000000000000000000000000000000 P 451334be41054bb498861a2fadb117a5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "451334be41054bb498861a2fadb117a5" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46791 } }
I20260811 11:55:45.222244 12357 sys_catalog.cc:564] T 00000000000000000000000000000000 P 451334be41054bb498861a2fadb117a5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:55:45.223528 12365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 451334be41054bb498861a2fadb117a5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 451334be41054bb498861a2fadb117a5. Latest consensus state: current_term: 1 leader_uuid: "451334be41054bb498861a2fadb117a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "451334be41054bb498861a2fadb117a5" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46791 } } }
I20260811 11:55:45.223532 12364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 451334be41054bb498861a2fadb117a5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "451334be41054bb498861a2fadb117a5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "451334be41054bb498861a2fadb117a5" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46791 } } }
I20260811 11:55:45.223693 12365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 451334be41054bb498861a2fadb117a5 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:45.223696 12364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 451334be41054bb498861a2fadb117a5 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:45.224109 12372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:55:45.226903 12372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:55:45.232157 12372 catalog_manager.cc:1347] Generated new cluster ID: f3495082ebbf484aa6ce56d4b9680a0a
I20260811 11:55:45.232221 12372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:55:45.243813 12372 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:55:45.244772 12372 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:55:45.252301 12372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 451334be41054bb498861a2fadb117a5: Generated new TSK 0
I20260811 11:55:45.252961 12372 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 11:55:45.278035 12389 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 11:55:45.278232 12389 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 11:55:45.278707 12360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:45.261968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41374 (local address 127.6.143.190:46791)
0811 11:55:45.262074 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:45.262079 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:45.262278 (+ 199us) server_negotiation.cc:407] Connection header received
0811 11:55:45.262469 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:45.262473 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:45.262529 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:45.262644 (+ 115us) server_negotiation.cc:226] Negotiated authn=JWT
0811 11:55:45.262881 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:45.263677 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:45.264584 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:45.264941 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:45.265045 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:45.265343 (+ 298us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 11:55:45.278326 (+ 12983us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44901/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 11:55:45.278565 (+ 239us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41374: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44901/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":27}
W20260811 11:55:45.278782 12360 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41374: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44901/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 11:55:45.280049 12388 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:55:45.261803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46791 (local address 127.0.0.1:41374)
0811 11:55:45.262190 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:45.262202 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:45.262308 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:45.262661 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:45.262663 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:45.262670 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0811 11:55:45.262772 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:45.262777 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:45.263784 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:45.263787 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:45.264466 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:45.264475 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:45.265055 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:45.265057 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:45.265122 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:45.265166 (+ 44us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 11:55:45.278512 (+ 13346us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44901/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 11:55:45.278769 (+ 257us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.143.190:46791: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44901/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":117,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.6.143.190:46791: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44901/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 11:55:45.280730 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 12285
2026-08-11T11:55:45Z chronyd exiting
I20260811 11:55:45.321349 6718 test_util.cc:175] -----------------------------------------------
I20260811 11:55:45.321432 6718 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786449319379472-6718-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (309 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 11:55:45.322496 6718 mini_oidc.cc:152] Starting JWKS server
I20260811 11:55:45.323354 6718 webserver.cc:460] Webserver started at https://127.0.0.1:40597/ using document root <none> and password file <none>
I20260811 11:55:45.323477 6718 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 11:55:45.323781 6718 webserver.cc:460] Webserver started at http://127.0.0.1:45805/ using document root <none> and password file <none>
2026-08-11T11:55:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:55:45Z Disabled control of system clock
I20260811 11:55:45.333606 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:37763
--webserver_interface=127.6.143.190
--webserver_port=0
--builtin_ntp_servers=127.6.143.148:36537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:37763
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40597/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449319379472-6718-0/testchainca.cert with env {}
W20260811 11:55:45.445082 12395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:45.445336 12395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:45.445394 12395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:45.449285 12395 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:55:45.449333 12395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:45.449348 12395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:45.449364 12395 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:55:45.449379 12395 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 11:55:45.449395 12395 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 11:55:45.449409 12395 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40597/jwks.json
I20260811 11:55:45.454572 12395 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:36537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:37763
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:37763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40597/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449319379472-6718-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:45.455500 12395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:45.456453 12395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:45.467144 12401 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:45.467269 12403 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:45.467459 12395 server_base.cc:1034] running on GCE node
W20260811 11:55:45.467691 12400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:45.468321 12395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:45.469405 12395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:45.470588 12395 hybrid_clock.cc:648] HybridClock initialized: now 1786449345470562 us; error 40 us; skew 500 ppm
I20260811 11:55:45.472497 12395 webserver.cc:460] Webserver started at http://127.6.143.190:38129/ using document root <none> and password file <none>
I20260811 11:55:45.473081 12395 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:45.473145 12395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:45.473397 12395 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:45.475445 12395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "b8061b48ad9948c09c303c43b433cc41"
format_stamp: "Formatted at 2026-08-11 11:55:45 on dist-test-slave-zkpd"
server_key: "27d62044c602ff392557ada76b81f700"
server_key_iv: "40b6681b1984b40b68b18b72e7143260"
server_key_version: "encryptionkey@0"
I20260811 11:55:45.476079 12395 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "b8061b48ad9948c09c303c43b433cc41"
format_stamp: "Formatted at 2026-08-11 11:55:45 on dist-test-slave-zkpd"
server_key: "27d62044c602ff392557ada76b81f700"
server_key_iv: "40b6681b1984b40b68b18b72e7143260"
server_key_version: "encryptionkey@0"
I20260811 11:55:45.479758 12395 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 11:55:45.482218 12409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:45.483394 12395 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:45.483520 12395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "b8061b48ad9948c09c303c43b433cc41"
format_stamp: "Formatted at 2026-08-11 11:55:45 on dist-test-slave-zkpd"
server_key: "27d62044c602ff392557ada76b81f700"
server_key_iv: "40b6681b1984b40b68b18b72e7143260"
server_key_version: "encryptionkey@0"
I20260811 11:55:45.483621 12395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:45.505249 12395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:45.505918 12395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:45.506108 12395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:45.513104 12395 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:37763
I20260811 11:55:45.513114 12464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:37763 every 8 connection(s)
I20260811 11:55:45.514057 12395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:55:45.517273 12465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:45.519527 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 12395
I20260811 11:55:45.519629 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:55:45.519958 6718 external_mini_cluster.cc:1428] Setting key 0dfc0a6eec28d5130f7d878d41abdd2a
I20260811 11:55:45.523455 12465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8061b48ad9948c09c303c43b433cc41: Bootstrap starting.
I20260811 11:55:45.526084 12465 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8061b48ad9948c09c303c43b433cc41: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:45.526909 12465 log.cc:824] T 00000000000000000000000000000000 P b8061b48ad9948c09c303c43b433cc41: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:45.529053 12465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8061b48ad9948c09c303c43b433cc41: No bootstrap required, opened a new log
I20260811 11:55:45.530337 12468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:45.521296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60918 (local address 127.6.143.190:37763)
0811 11:55:45.521971 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:45.522002 (+ 31us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:45.522045 (+ 43us) server_negotiation.cc:407] Connection header received
0811 11:55:45.522880 (+ 835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:45.522904 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:45.523377 (+ 473us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:45.523751 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:45.524232 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:45.525704 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:45.526596 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:45.527301 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:45.527422 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:45.528379 (+ 957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:45.528385 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:45.528410 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:55:45.528873 (+ 463us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:45.528944 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:55:45.528970 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:45.529065 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:45.529208 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 11:55:45.529485 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":107,"threads_started":1}
I20260811 11:55:45.533272 6718 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:55:45.533286 12465 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8061b48ad9948c09c303c43b433cc41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8061b48ad9948c09c303c43b433cc41" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 37763 } }
I20260811 11:55:45.533529 12465 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8061b48ad9948c09c303c43b433cc41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:45.533605 12465 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8061b48ad9948c09c303c43b433cc41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8061b48ad9948c09c303c43b433cc41, State: Initialized, Role: FOLLOWER
I20260811 11:55:45.534009 12465 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8061b48ad9948c09c303c43b433cc41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8061b48ad9948c09c303c43b433cc41" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 37763 } }
I20260811 11:55:45.534124 12465 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8061b48ad9948c09c303c43b433cc41 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:55:45.534204 12465 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8061b48ad9948c09c303c43b433cc41 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:55:45.534319 12465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8061b48ad9948c09c303c43b433cc41 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:45.535236 12465 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8061b48ad9948c09c303c43b433cc41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8061b48ad9948c09c303c43b433cc41" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 37763 } }
I20260811 11:55:45.535518 12465 leader_election.cc:302] T 00000000000000000000000000000000 P b8061b48ad9948c09c303c43b433cc41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8061b48ad9948c09c303c43b433cc41; no voters:
I20260811 11:55:45.535749 12465 leader_election.cc:288] T 00000000000000000000000000000000 P b8061b48ad9948c09c303c43b433cc41 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:55:45.535879 12470 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8061b48ad9948c09c303c43b433cc41 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:45.536120 12470 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8061b48ad9948c09c303c43b433cc41 [term 1 LEADER]: Becoming Leader. State: Replica: b8061b48ad9948c09c303c43b433cc41, State: Running, Role: LEADER
I20260811 11:55:45.536432 12470 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8061b48ad9948c09c303c43b433cc41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8061b48ad9948c09c303c43b433cc41" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 37763 } }
I20260811 11:55:45.536871 12465 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8061b48ad9948c09c303c43b433cc41 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:55:45.538127 12473 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8061b48ad9948c09c303c43b433cc41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8061b48ad9948c09c303c43b433cc41. Latest consensus state: current_term: 1 leader_uuid: "b8061b48ad9948c09c303c43b433cc41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8061b48ad9948c09c303c43b433cc41" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 37763 } } }
I20260811 11:55:45.538127 12471 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8061b48ad9948c09c303c43b433cc41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8061b48ad9948c09c303c43b433cc41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8061b48ad9948c09c303c43b433cc41" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 37763 } } }
I20260811 11:55:45.538295 12471 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8061b48ad9948c09c303c43b433cc41 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:45.538295 12473 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8061b48ad9948c09c303c43b433cc41 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:45.538816 12484 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:55:45.541510 12484 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:55:45.546604 12484 catalog_manager.cc:1347] Generated new cluster ID: c2ab84a44b5d4f6eb0737c651f48e510
I20260811 11:55:45.546677 12484 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:55:45.586004 12484 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:55:45.587054 12484 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:55:45.599015 12484 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8061b48ad9948c09c303c43b433cc41: Generated new TSK 0
I20260811 11:55:45.599846 12484 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 11:55:45.638974 12499 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 11:55:45.639075 12499 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 11:55:45.639636 12468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:45.617080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60934 (local address 127.6.143.190:37763)
0811 11:55:45.617227 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:45.617232 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:45.617428 (+ 196us) server_negotiation.cc:407] Connection header received
0811 11:55:45.617604 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:45.617607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:45.617655 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:45.617739 (+ 84us) server_negotiation.cc:226] Negotiated authn=JWT
0811 11:55:45.618035 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:45.618532 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:45.619754 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:45.620017 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:45.620095 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:45.620422 (+ 327us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 11:55:45.639282 (+ 18860us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40597/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 11:55:45.639515 (+ 233us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:60934: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40597/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 11:55:45.639720 12468 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:60934: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40597/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 11:55:45.639858 12498 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:55:45.617003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:37763 (local address 127.0.0.1:60934)
0811 11:55:45.617329 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:45.617346 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:45.617465 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:45.617775 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:45.617778 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:45.617787 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0811 11:55:45.617924 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:45.617931 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:45.618640 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:45.618644 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:45.619636 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:45.619645 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:45.620125 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:45.620128 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:45.620218 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:45.620248 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 11:55:45.639477 (+ 19229us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40597/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 11:55:45.639673 (+ 196us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.143.190:37763: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40597/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":121,"threads_started":1}
I20260811 11:55:45.640600 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 12395
2026-08-11T11:55:45Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (361 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 11:55:45.684394 6718 mini_oidc.cc:152] Starting JWKS server
I20260811 11:55:45.685252 6718 webserver.cc:460] Webserver started at https://127.0.0.1:35241/ using document root <none> and password file <none>
I20260811 11:55:45.685364 6718 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 11:55:45.685637 6718 webserver.cc:460] Webserver started at http://127.0.0.1:41783/ using document root <none> and password file <none>
2026-08-11T11:55:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:55:45Z Disabled control of system clock
I20260811 11:55:45.695729 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:37635
--webserver_interface=127.6.143.190
--webserver_port=0
--builtin_ntp_servers=127.6.143.148:46353
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:37635
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35241/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 11:55:45.806038 12507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:45.806274 12507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:45.806311 12507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:45.810185 12507 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:55:45.810231 12507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:45.810246 12507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:45.810262 12507 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:55:45.810277 12507 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 11:55:45.810292 12507 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 11:55:45.810305 12507 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35241/jwks.json
I20260811 11:55:45.815007 12507 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:46353
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:37635
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:37635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35241/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:45.815945 12507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:45.817205 12507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:45.826323 12515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:45.826354 12513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:45.826551 12507 server_base.cc:1034] running on GCE node
W20260811 11:55:45.826313 12512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:45.827215 12507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:45.828388 12507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:45.829579 12507 hybrid_clock.cc:648] HybridClock initialized: now 1786449345829559 us; error 30 us; skew 500 ppm
I20260811 11:55:45.835418 12507 webserver.cc:460] Webserver started at http://127.6.143.190:45285/ using document root <none> and password file <none>
I20260811 11:55:45.836120 12507 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:45.836213 12507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:45.836457 12507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:45.838490 12507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "04d1c2b5860a4261a15a235bae1cf8f2"
format_stamp: "Formatted at 2026-08-11 11:55:45 on dist-test-slave-zkpd"
server_key: "5ffa1a9db491285624f79a2711a9b0fe"
server_key_iv: "b5f47fe25d18aa1b5b0d453b67c64c14"
server_key_version: "encryptionkey@0"
I20260811 11:55:45.839057 12507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "04d1c2b5860a4261a15a235bae1cf8f2"
format_stamp: "Formatted at 2026-08-11 11:55:45 on dist-test-slave-zkpd"
server_key: "5ffa1a9db491285624f79a2711a9b0fe"
server_key_iv: "b5f47fe25d18aa1b5b0d453b67c64c14"
server_key_version: "encryptionkey@0"
I20260811 11:55:45.842723 12507 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260811 11:55:45.845105 12522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:45.846243 12507 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 11:55:45.846395 12507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "04d1c2b5860a4261a15a235bae1cf8f2"
format_stamp: "Formatted at 2026-08-11 11:55:45 on dist-test-slave-zkpd"
server_key: "5ffa1a9db491285624f79a2711a9b0fe"
server_key_iv: "b5f47fe25d18aa1b5b0d453b67c64c14"
server_key_version: "encryptionkey@0"
I20260811 11:55:45.846506 12507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:45.869581 12507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:45.870240 12507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:45.870440 12507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:45.877166 12507 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:37635
I20260811 11:55:45.877170 12583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:37635 every 8 connection(s)
I20260811 11:55:45.878094 12507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:55:45.880942 12584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:45.881431 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 12507
I20260811 11:55:45.881573 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:55:45.881845 6718 external_mini_cluster.cc:1428] Setting key 75d030b79ebb027c0eddb00d3b839ad4
I20260811 11:55:45.887677 12584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04d1c2b5860a4261a15a235bae1cf8f2: Bootstrap starting.
I20260811 11:55:45.890264 12584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04d1c2b5860a4261a15a235bae1cf8f2: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:45.891208 12584 log.cc:824] T 00000000000000000000000000000000 P 04d1c2b5860a4261a15a235bae1cf8f2: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:45.892257 12587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:45.883224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56064 (local address 127.6.143.190:37635)
0811 11:55:45.883937 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:45.883957 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:45.884002 (+ 45us) server_negotiation.cc:407] Connection header received
0811 11:55:45.884830 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:45.884853 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:45.885347 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:45.885732 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:45.886194 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:45.887640 (+ 1446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:45.888601 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:45.889296 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:45.889463 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:45.890239 (+ 776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:45.890245 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:45.890261 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:55:45.890774 (+ 513us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:45.890841 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:55:45.890867 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:45.890972 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:45.891116 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 11:55:45.891378 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":111,"threads_started":1}
I20260811 11:55:45.893316 12584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04d1c2b5860a4261a15a235bae1cf8f2: No bootstrap required, opened a new log
I20260811 11:55:45.895031 6718 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:55:45.897286 12584 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04d1c2b5860a4261a15a235bae1cf8f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04d1c2b5860a4261a15a235bae1cf8f2" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 37635 } }
I20260811 11:55:45.897491 12584 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04d1c2b5860a4261a15a235bae1cf8f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:45.897598 12584 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04d1c2b5860a4261a15a235bae1cf8f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04d1c2b5860a4261a15a235bae1cf8f2, State: Initialized, Role: FOLLOWER
I20260811 11:55:45.898052 12584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04d1c2b5860a4261a15a235bae1cf8f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04d1c2b5860a4261a15a235bae1cf8f2" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 37635 } }
I20260811 11:55:45.898195 12584 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04d1c2b5860a4261a15a235bae1cf8f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:55:45.898280 12584 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04d1c2b5860a4261a15a235bae1cf8f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:55:45.898399 12584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04d1c2b5860a4261a15a235bae1cf8f2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:45.899241 12584 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04d1c2b5860a4261a15a235bae1cf8f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04d1c2b5860a4261a15a235bae1cf8f2" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 37635 } }
I20260811 11:55:45.899581 12584 leader_election.cc:302] T 00000000000000000000000000000000 P 04d1c2b5860a4261a15a235bae1cf8f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04d1c2b5860a4261a15a235bae1cf8f2; no voters:
I20260811 11:55:45.899874 12584 leader_election.cc:288] T 00000000000000000000000000000000 P 04d1c2b5860a4261a15a235bae1cf8f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:55:45.900028 12589 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04d1c2b5860a4261a15a235bae1cf8f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:45.900281 12589 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04d1c2b5860a4261a15a235bae1cf8f2 [term 1 LEADER]: Becoming Leader. State: Replica: 04d1c2b5860a4261a15a235bae1cf8f2, State: Running, Role: LEADER
I20260811 11:55:45.900643 12589 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04d1c2b5860a4261a15a235bae1cf8f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04d1c2b5860a4261a15a235bae1cf8f2" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 37635 } }
I20260811 11:55:45.901165 12584 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04d1c2b5860a4261a15a235bae1cf8f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:55:45.902539 12591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04d1c2b5860a4261a15a235bae1cf8f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04d1c2b5860a4261a15a235bae1cf8f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04d1c2b5860a4261a15a235bae1cf8f2" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 37635 } } }
I20260811 11:55:45.902539 12592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04d1c2b5860a4261a15a235bae1cf8f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04d1c2b5860a4261a15a235bae1cf8f2. Latest consensus state: current_term: 1 leader_uuid: "04d1c2b5860a4261a15a235bae1cf8f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04d1c2b5860a4261a15a235bae1cf8f2" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 37635 } } }
I20260811 11:55:45.902693 12591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04d1c2b5860a4261a15a235bae1cf8f2 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:45.902693 12592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04d1c2b5860a4261a15a235bae1cf8f2 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:45.903136 12600 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:55:45.906000 12600 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:55:45.911135 12600 catalog_manager.cc:1347] Generated new cluster ID: 8818f78d4f174fb8a439856755e33200
I20260811 11:55:45.911199 12600 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:55:45.940828 12600 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:55:45.942085 12600 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:55:45.948302 12600 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04d1c2b5860a4261a15a235bae1cf8f2: Generated new TSK 0
I20260811 11:55:45.949083 12600 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:55:45.963348 12587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:45.957698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56066 (local address 127.6.143.190:37635)
0811 11:55:45.957834 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:45.957839 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:45.958065 (+ 226us) server_negotiation.cc:407] Connection header received
0811 11:55:45.958211 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:45.958213 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:45.958259 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:45.958344 (+ 85us) server_negotiation.cc:226] Negotiated authn=JWT
0811 11:55:45.958610 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:45.959142 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:45.960413 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:45.960630 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:45.960709 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:45.961047 (+ 338us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 11:55:45.962994 (+ 1947us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35241/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 11:55:45.963218 (+ 224us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:56066: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35241/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 11:55:45.963421 12587 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:56066: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35241/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 11:55:45.965232 12616 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:55:45.957628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:37635 (local address 127.0.0.1:56066)
0811 11:55:45.957971 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:45.957987 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:45.958104 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:45.958375 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:45.958378 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:45.958386 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0811 11:55:45.958508 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:45.958514 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:45.959258 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:45.959262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:45.960300 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:45.960312 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:45.960748 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:45.960752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:45.960845 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:45.960876 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 11:55:45.963176 (+ 2300us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35241/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 11:55:45.965064 (+ 1888us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.143.190:37635: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35241/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":1675,"thread_start_us":124,"threads_started":1}
I20260811 11:55:45.966145 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 12507
E20260811 11:55:45.977838 12617 webserver.cc:533] Webserver: sslize failed:
E20260811 11:55:45.977905 12617 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-11T11:55:45Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (325 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T11:55:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:55:46Z Disabled control of system clock
I20260811 11:55:46.019302 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:43453
--webserver_interface=127.6.143.190
--webserver_port=0
--builtin_ntp_servers=127.6.143.148:33531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:43453
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449319379472-6718-0/testchainca.cert with env {}
W20260811 11:55:46.130358 12621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:46.130610 12621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:46.130650 12621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:46.134709 12621 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:55:46.134760 12621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:46.134776 12621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:46.134828 12621 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:55:46.134863 12621 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 11:55:46.134899 12621 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 11:55:46.134948 12621 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 11:55:46.140093 12621 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:33531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:43453
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:43453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449319379472-6718-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:46.141063 12621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:46.142047 12621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:46.150322 12626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:46.150378 12621 server_base.cc:1034] running on GCE node
W20260811 11:55:46.150331 12628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:46.150322 12631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:46.151152 12621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:46.152279 12621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:46.153442 12621 hybrid_clock.cc:648] HybridClock initialized: now 1786449346153415 us; error 48 us; skew 500 ppm
I20260811 11:55:46.155268 12621 webserver.cc:460] Webserver started at http://127.6.143.190:33673/ using document root <none> and password file <none>
I20260811 11:55:46.155833 12621 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:46.155967 12621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:46.156205 12621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:46.158165 12621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "74b2055291134945bf0ad17921975d6e"
format_stamp: "Formatted at 2026-08-11 11:55:46 on dist-test-slave-zkpd"
server_key: "ff2ac44a99cac9463785939914cac312"
server_key_iv: "7e743238b7630a476a7342a0ae985d23"
server_key_version: "encryptionkey@0"
I20260811 11:55:46.158726 12621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "74b2055291134945bf0ad17921975d6e"
format_stamp: "Formatted at 2026-08-11 11:55:46 on dist-test-slave-zkpd"
server_key: "ff2ac44a99cac9463785939914cac312"
server_key_iv: "7e743238b7630a476a7342a0ae985d23"
server_key_version: "encryptionkey@0"
I20260811 11:55:46.162290 12621 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 11:55:46.164734 12637 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:46.165853 12621 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:55:46.165995 12621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "74b2055291134945bf0ad17921975d6e"
format_stamp: "Formatted at 2026-08-11 11:55:46 on dist-test-slave-zkpd"
server_key: "ff2ac44a99cac9463785939914cac312"
server_key_iv: "7e743238b7630a476a7342a0ae985d23"
server_key_version: "encryptionkey@0"
I20260811 11:55:46.166103 12621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:46.182089 12621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:46.182732 12621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:46.182936 12621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:46.190022 12621 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:43453
I20260811 11:55:46.190053 12695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:43453 every 8 connection(s)
I20260811 11:55:46.190917 12621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:55:46.194068 12696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:46.194823 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 12621
I20260811 11:55:46.194977 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:55:46.195268 6718 external_mini_cluster.cc:1428] Setting key d500ee60b3e0e36c1dafb9b33ee0e938
I20260811 11:55:46.201022 12696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74b2055291134945bf0ad17921975d6e: Bootstrap starting.
I20260811 11:55:46.203368 12696 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74b2055291134945bf0ad17921975d6e: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:46.204138 12696 log.cc:824] T 00000000000000000000000000000000 P 74b2055291134945bf0ad17921975d6e: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:46.204319 12699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:46.196504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57486 (local address 127.6.143.190:43453)
0811 11:55:46.197068 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:46.197082 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:46.197116 (+ 34us) server_negotiation.cc:407] Connection header received
0811 11:55:46.197921 (+ 805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:46.197940 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:46.198313 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:46.198661 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:46.199065 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:46.200075 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:46.200953 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:46.201465 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:46.201639 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:46.202527 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:46.202531 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:46.202543 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:55:46.202907 (+ 364us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:46.202962 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:55:46.202983 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:46.203087 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:46.203249 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 11:55:46.203546 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":94,"threads_started":1}
I20260811 11:55:46.206321 12696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74b2055291134945bf0ad17921975d6e: No bootstrap required, opened a new log
I20260811 11:55:46.207188 6718 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:55:46.210451 12696 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74b2055291134945bf0ad17921975d6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b2055291134945bf0ad17921975d6e" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 43453 } }
I20260811 11:55:46.210747 12696 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74b2055291134945bf0ad17921975d6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:46.210813 12696 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74b2055291134945bf0ad17921975d6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74b2055291134945bf0ad17921975d6e, State: Initialized, Role: FOLLOWER
I20260811 11:55:46.211217 12696 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74b2055291134945bf0ad17921975d6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b2055291134945bf0ad17921975d6e" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 43453 } }
I20260811 11:55:46.211354 12696 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74b2055291134945bf0ad17921975d6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:55:46.211418 12696 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74b2055291134945bf0ad17921975d6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:55:46.211510 12696 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74b2055291134945bf0ad17921975d6e [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:46.212436 12696 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74b2055291134945bf0ad17921975d6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b2055291134945bf0ad17921975d6e" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 43453 } }
I20260811 11:55:46.212811 12696 leader_election.cc:302] T 00000000000000000000000000000000 P 74b2055291134945bf0ad17921975d6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74b2055291134945bf0ad17921975d6e; no voters:
I20260811 11:55:46.213128 12696 leader_election.cc:288] T 00000000000000000000000000000000 P 74b2055291134945bf0ad17921975d6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:55:46.213248 12701 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74b2055291134945bf0ad17921975d6e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:46.213501 12701 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74b2055291134945bf0ad17921975d6e [term 1 LEADER]: Becoming Leader. State: Replica: 74b2055291134945bf0ad17921975d6e, State: Running, Role: LEADER
I20260811 11:55:46.213835 12701 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74b2055291134945bf0ad17921975d6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b2055291134945bf0ad17921975d6e" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 43453 } }
I20260811 11:55:46.214494 12696 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74b2055291134945bf0ad17921975d6e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:55:46.215582 12702 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74b2055291134945bf0ad17921975d6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74b2055291134945bf0ad17921975d6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b2055291134945bf0ad17921975d6e" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 43453 } } }
I20260811 11:55:46.215719 12702 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74b2055291134945bf0ad17921975d6e [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:46.215572 12703 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74b2055291134945bf0ad17921975d6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74b2055291134945bf0ad17921975d6e. Latest consensus state: current_term: 1 leader_uuid: "74b2055291134945bf0ad17921975d6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74b2055291134945bf0ad17921975d6e" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 43453 } } }
I20260811 11:55:46.215808 12703 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74b2055291134945bf0ad17921975d6e [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:46.216197 12712 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:55:46.219431 12712 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:55:46.224982 12712 catalog_manager.cc:1347] Generated new cluster ID: 9fa52702a1e543998118a7dfeb0c483e
I20260811 11:55:46.225077 12712 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:55:46.240932 12712 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:55:46.241768 12712 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:55:46.251559 12712 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74b2055291134945bf0ad17921975d6e: Generated new TSK 0
I20260811 11:55:46.252242 12712 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:55:46.382938 12699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:46.262147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57492 (local address 127.6.143.190:43453)
0811 11:55:46.262279 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:46.262284 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:46.262438 (+ 154us) server_negotiation.cc:407] Connection header received
0811 11:55:46.262567 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:46.262570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:46.262624 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:46.262736 (+ 112us) server_negotiation.cc:226] Negotiated authn=JWT
0811 11:55:46.262946 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:46.263777 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:46.264659 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:46.265021 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:46.265127 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:46.265395 (+ 268us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 11:55:46.382515 (+117120us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 11:55:46.382805 (+ 290us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57492: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 11:55:46.383018 12699 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57492: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 11:55:46.383203 12728 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:55:46.262023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:43453 (local address 127.0.0.1:57492)
0811 11:55:46.262350 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:46.262363 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:46.262464 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:46.262739 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:46.262741 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:46.262747 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0811 11:55:46.262845 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:46.262854 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:46.263881 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:46.263883 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:46.264544 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:46.264554 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:46.265140 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:46.265143 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:46.265207 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:46.265230 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 11:55:46.382757 (+117527us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 11:55:46.382953 (+ 196us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.143.190:43453: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":114,"threads_started":1}
I20260811 11:55:46.384210 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 12621
2026-08-11T11:55:46Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (396 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (133 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (133 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:55:48 dist-test-slave-zkpd krb5kdc[12740](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:55:48 dist-test-slave-zkpd krb5kdc[12740](info): set up 2 sockets
Aug 11 11:55:48 dist-test-slave-zkpd krb5kdc[12740](info): commencing operation
krb5kdc: starting...
W20260811 11:55:49.060881 6718 mini_kdc.cc:121] Time spent starting KDC: real 2.386s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:55:49 dist-test-slave-zkpd krb5kdc[12740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:55:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:55:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
I20260811 11:55:49.186595 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:44703
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:37599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:44703
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:49.312728 12754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:49.312985 12754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:49.313024 12754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:49.317126 12754 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:55:49.317176 12754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:49.317195 12754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:49.317212 12754 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:55:49.317227 12754 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:55:49.322382 12754 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:37599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:44703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:44703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:49.323315 12754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:49.324306 12754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:49.332672 12759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:49.332672 12761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:49.332667 12764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:49.332770 12754 server_base.cc:1034] running on GCE node
I20260811 11:55:49.333518 12754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:49.334578 12754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:49.335763 12754 hybrid_clock.cc:648] HybridClock initialized: now 1786449349335728 us; error 49 us; skew 500 ppm
Aug 11 11:55:49 dist-test-slave-zkpd krb5kdc[12740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449349, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:49.338524 12754 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:55:49.339411 12754 webserver.cc:460] Webserver started at http://127.6.143.190:35759/ using document root <none> and password file <none>
I20260811 11:55:49.340027 12754 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:49.340117 12754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:49.340346 12754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:49.342402 12754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "99fc577ed07f4b17ba1656625829f9c3"
format_stamp: "Formatted at 2026-08-11 11:55:49 on dist-test-slave-zkpd"
server_key: "0bd876e55d63394970b8f7412634f1ec"
server_key_iv: "fdeea913eeed6f18f8789bf600462f6c"
server_key_version: "encryptionkey@0"
I20260811 11:55:49.342965 12754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "99fc577ed07f4b17ba1656625829f9c3"
format_stamp: "Formatted at 2026-08-11 11:55:49 on dist-test-slave-zkpd"
server_key: "0bd876e55d63394970b8f7412634f1ec"
server_key_iv: "fdeea913eeed6f18f8789bf600462f6c"
server_key_version: "encryptionkey@0"
I20260811 11:55:49.346680 12754 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260811 11:55:49.349155 12772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:49.350346 12754 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 11:55:49.350483 12754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "99fc577ed07f4b17ba1656625829f9c3"
format_stamp: "Formatted at 2026-08-11 11:55:49 on dist-test-slave-zkpd"
server_key: "0bd876e55d63394970b8f7412634f1ec"
server_key_iv: "fdeea913eeed6f18f8789bf600462f6c"
server_key_version: "encryptionkey@0"
I20260811 11:55:49.350589 12754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:49.362382 12754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:49.365564 12754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:49.365760 12754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:49.372908 12754 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:44703
I20260811 11:55:49.373039 12827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:44703 every 8 connection(s)
I20260811 11:55:49.373905 12754 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:55:49.376926 12828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:49.382833 12828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99fc577ed07f4b17ba1656625829f9c3: Bootstrap starting.
I20260811 11:55:49.383131 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 12754
I20260811 11:55:49.383281 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:55:49.383553 6718 external_mini_cluster.cc:1428] Setting key 21f25ccf774913635a92dd6b0c1edbc6
I20260811 11:55:49.385747 12828 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 99fc577ed07f4b17ba1656625829f9c3: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:49.386585 12828 log.cc:824] T 00000000000000000000000000000000 P 99fc577ed07f4b17ba1656625829f9c3: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:49.388644 12828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 99fc577ed07f4b17ba1656625829f9c3: No bootstrap required, opened a new log
Aug 11 11:55:49 dist-test-slave-zkpd krb5kdc[12740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449349, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:49.392787 12828 raft_consensus.cc:348] T 00000000000000000000000000000000 P 99fc577ed07f4b17ba1656625829f9c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99fc577ed07f4b17ba1656625829f9c3" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 44703 } }
I20260811 11:55:49.393013 12828 raft_consensus.cc:374] T 00000000000000000000000000000000 P 99fc577ed07f4b17ba1656625829f9c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:49.393064 12828 raft_consensus.cc:729] T 00000000000000000000000000000000 P 99fc577ed07f4b17ba1656625829f9c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99fc577ed07f4b17ba1656625829f9c3, State: Initialized, Role: FOLLOWER
I20260811 11:55:49.393560 12828 consensus_queue.cc:260] T 00000000000000000000000000000000 P 99fc577ed07f4b17ba1656625829f9c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99fc577ed07f4b17ba1656625829f9c3" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 44703 } }
I20260811 11:55:49.393708 12828 raft_consensus.cc:388] T 00000000000000000000000000000000 P 99fc577ed07f4b17ba1656625829f9c3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:55:49.393826 12828 raft_consensus.cc:482] T 00000000000000000000000000000000 P 99fc577ed07f4b17ba1656625829f9c3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:55:49.393947 12828 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 99fc577ed07f4b17ba1656625829f9c3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:49.394876 12828 raft_consensus.cc:504] T 00000000000000000000000000000000 P 99fc577ed07f4b17ba1656625829f9c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99fc577ed07f4b17ba1656625829f9c3" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 44703 } }
I20260811 11:55:49.395256 12828 leader_election.cc:302] T 00000000000000000000000000000000 P 99fc577ed07f4b17ba1656625829f9c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 99fc577ed07f4b17ba1656625829f9c3; no voters:
I20260811 11:55:49.395638 12828 leader_election.cc:288] T 00000000000000000000000000000000 P 99fc577ed07f4b17ba1656625829f9c3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:55:49.395722 12833 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 99fc577ed07f4b17ba1656625829f9c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:49.395967 12833 raft_consensus.cc:686] T 00000000000000000000000000000000 P 99fc577ed07f4b17ba1656625829f9c3 [term 1 LEADER]: Becoming Leader. State: Replica: 99fc577ed07f4b17ba1656625829f9c3, State: Running, Role: LEADER
I20260811 11:55:49.396356 12833 consensus_queue.cc:237] T 00000000000000000000000000000000 P 99fc577ed07f4b17ba1656625829f9c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99fc577ed07f4b17ba1656625829f9c3" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 44703 } }
I20260811 11:55:49.397065 12828 sys_catalog.cc:564] T 00000000000000000000000000000000 P 99fc577ed07f4b17ba1656625829f9c3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:55:49.398114 12834 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99fc577ed07f4b17ba1656625829f9c3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "99fc577ed07f4b17ba1656625829f9c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99fc577ed07f4b17ba1656625829f9c3" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 44703 } } }
I20260811 11:55:49.398268 12834 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99fc577ed07f4b17ba1656625829f9c3 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:49.398360 12833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 99fc577ed07f4b17ba1656625829f9c3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 99fc577ed07f4b17ba1656625829f9c3. Latest consensus state: current_term: 1 leader_uuid: "99fc577ed07f4b17ba1656625829f9c3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "99fc577ed07f4b17ba1656625829f9c3" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 44703 } } }
I20260811 11:55:49.398451 12833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 99fc577ed07f4b17ba1656625829f9c3 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:49.398286 12831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:49.384970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40704 (local address 127.6.143.190:44703)
0811 11:55:49.385426 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:49.385442 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:49.385478 (+ 36us) server_negotiation.cc:407] Connection header received
0811 11:55:49.386168 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:49.386188 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:49.386528 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:49.386856 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:49.387659 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:49.388676 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:49.389533 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:49.389982 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:49.390159 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:49.392919 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:49.392925 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:49.392938 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:49.392965 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:49.394918 (+ 1953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:49.395382 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:49.395388 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:49.395396 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:49.395532 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:49.395939 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:49.395945 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:49.395948 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:49.396491 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:49.396705 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:49.396987 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 11:55:49.397308 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":143,"threads_started":1}
W20260811 11:55:49.402477 12849 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 99fc577ed07f4b17ba1656625829f9c3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 11:55:49.402542 12849 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 11:55:49.402622 12842 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:55:49.403311 12842 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:55:49.409314 12842 catalog_manager.cc:1347] Generated new cluster ID: c2bef381caa349d3854bde655dac310d
I20260811 11:55:49.409387 12842 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:55:49.427752 12842 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:55:49.429051 12842 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:55:49.437445 12842 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 99fc577ed07f4b17ba1656625829f9c3: Generated new TSK 0
I20260811 11:55:49.438264 12842 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
I20260811 11:55:49.472769 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:0
--local_ip_for_outbound_sockets=127.6.143.129
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:44703
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:37599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:49.584993 12856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:49.585273 12856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:49.585371 12856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:49.589478 12856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:49.589556 12856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:49.589643 12856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:55:49.594647 12856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:37599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:44703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:49.595614 12856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:49.596657 12856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:49.606247 12862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:49.606225 12861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:49.606456 12856 server_base.cc:1034] running on GCE node
W20260811 11:55:49.606218 12864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:49.606925 12856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:49.607455 12856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:49.608608 12856 hybrid_clock.cc:648] HybridClock initialized: now 1786449349608572 us; error 45 us; skew 500 ppm
Aug 11 11:55:49 dist-test-slave-zkpd krb5kdc[12740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449349, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:49.611264 12856 init.cc:377] Logged in from keytab as kudu/127.6.143.129@KRBTEST.COM (short username kudu)
I20260811 11:55:49.612185 12856 webserver.cc:460] Webserver started at http://127.6.143.129:46753/ using document root <none> and password file <none>
I20260811 11:55:49.612721 12856 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:49.612782 12856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:49.613022 12856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:49.615006 12856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-0/data/instance:
uuid: "4c369b534f0244248afd528579996be2"
format_stamp: "Formatted at 2026-08-11 11:55:49 on dist-test-slave-zkpd"
server_key: "b360c571cdf641c581b566ab9823cd88"
server_key_iv: "6f85c1482567cb857a5718e282ad7e92"
server_key_version: "encryptionkey@0"
I20260811 11:55:49.615564 12856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance:
uuid: "4c369b534f0244248afd528579996be2"
format_stamp: "Formatted at 2026-08-11 11:55:49 on dist-test-slave-zkpd"
server_key: "b360c571cdf641c581b566ab9823cd88"
server_key_iv: "6f85c1482567cb857a5718e282ad7e92"
server_key_version: "encryptionkey@0"
I20260811 11:55:49.619204 12856 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 11:55:49.621599 12872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:49.622851 12856 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 11:55:49.622946 12856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "4c369b534f0244248afd528579996be2"
format_stamp: "Formatted at 2026-08-11 11:55:49 on dist-test-slave-zkpd"
server_key: "b360c571cdf641c581b566ab9823cd88"
server_key_iv: "6f85c1482567cb857a5718e282ad7e92"
server_key_version: "encryptionkey@0"
I20260811 11:55:49.623027 12856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:49.640038 12856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:49.643275 12856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:49.643435 12856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:49.644021 12856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:55:49.644850 12856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:49.644904 12856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:49.644945 12856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:49.644961 12856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:49.653887 12856 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:42505
I20260811 11:55:49.653944 12994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:42505 every 8 connection(s)
I20260811 11:55:49.654913 12856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:55:49.659353 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 12856
I20260811 11:55:49.659494 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance
I20260811 11:55:49.659772 6718 external_mini_cluster.cc:1428] Setting key 994aef5be7dc6befab9f4c81b209e7a2
Aug 11 11:55:49 dist-test-slave-zkpd krb5kdc[12740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449349, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:49.669263 12831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:49.656833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:48405 (local address 127.6.143.190:44703)
0811 11:55:49.656988 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:49.656991 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:49.657869 (+ 878us) server_negotiation.cc:407] Connection header received
0811 11:55:49.659304 (+ 1435us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:49.659310 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:49.659382 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:49.659486 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:49.660825 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:49.661354 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:49.662479 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:49.662718 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:49.662831 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:49.665361 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:49.665369 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:49.665372 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:49.665397 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:49.666895 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:49.667590 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:49.667593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:49.667595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:49.667689 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:49.668132 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:49.668134 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:49.668135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:49.668306 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:49.668389 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:49.669043 (+ 654us) server_negotiation.cc:299] Negotiation successful
0811 11:55:49.669145 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 11:55:49.670428 12997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:49.657209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:44703 (local address 127.6.143.129:48405)
0811 11:55:49.657683 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:49.657749 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:49.659076 (+ 1327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:49.659668 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:49.659674 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:49.660124 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:49.660661 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:49.660676 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:49.661485 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:49.661495 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:49.662349 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:49.662359 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:49.662881 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:49.662885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:49.663052 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:49.663657 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:49.663673 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:49.665180 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:49.667029 (+ 1849us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:49.667037 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:49.667054 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:49.667459 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:49.667796 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:49.667802 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:49.667806 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:49.668029 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:49.668421 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:49.668430 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:49.668777 (+ 347us) client_negotiation.cc:769] Sending connection context
0811 11:55:49.668993 (+ 216us) client_negotiation.cc:240] Negotiation successful
0811 11:55:49.669355 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":119,"threads_started":1}
WARNING: no policy specified for kudu/127.6.143.130@KRBTEST.COM; defaulting to no policy
I20260811 11:55:49.671640 12995 heartbeater.cc:347] Connected to a master server at 127.6.143.190:44703
I20260811 11:55:49.671957 12995 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:49.672472 12995 heartbeater.cc:511] Master 127.6.143.190:44703 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.130@KRBTEST.COM" created.
I20260811 11:55:49.674142 12791 ts_manager.cc:194] Registered new tserver with Master: 4c369b534f0244248afd528579996be2 (127.6.143.129:42505)
I20260811 11:55:49.675381 12791 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.129@KRBTEST.COM'} at 127.6.143.129:48405
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
I20260811 11:55:49.689066 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:0
--local_ip_for_outbound_sockets=127.6.143.130
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:44703
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:37599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:49.801883 13000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:49.802139 13000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:49.802182 13000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:49.806190 13000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:49.806243 13000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:49.806298 13000 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:55:49.811384 13000 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:37599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:44703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:49.812345 13000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:49.813365 13000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:49.823058 13007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:49.823166 13000 server_base.cc:1034] running on GCE node
W20260811 11:55:49.823065 13006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:49.823058 13009 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:49.823794 13000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:49.824431 13000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:49.825589 13000 hybrid_clock.cc:648] HybridClock initialized: now 1786449349825555 us; error 46 us; skew 500 ppm
Aug 11 11:55:49 dist-test-slave-zkpd krb5kdc[12740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449349, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:49.828534 13000 init.cc:377] Logged in from keytab as kudu/127.6.143.130@KRBTEST.COM (short username kudu)
I20260811 11:55:49.829392 13000 webserver.cc:460] Webserver started at http://127.6.143.130:34479/ using document root <none> and password file <none>
I20260811 11:55:49.829945 13000 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:49.830006 13000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:49.830236 13000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:49.832330 13000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-1/data/instance:
uuid: "ce7151a62e2b414e8f4092cb5cbd6fb3"
format_stamp: "Formatted at 2026-08-11 11:55:49 on dist-test-slave-zkpd"
server_key: "41aae06ebf63227276e1a094e5d5c43f"
server_key_iv: "7c9821810f45c1c4dc9246efb8d2fa94"
server_key_version: "encryptionkey@0"
I20260811 11:55:49.832901 13000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance:
uuid: "ce7151a62e2b414e8f4092cb5cbd6fb3"
format_stamp: "Formatted at 2026-08-11 11:55:49 on dist-test-slave-zkpd"
server_key: "41aae06ebf63227276e1a094e5d5c43f"
server_key_iv: "7c9821810f45c1c4dc9246efb8d2fa94"
server_key_version: "encryptionkey@0"
I20260811 11:55:49.836818 13000 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 11:55:49.840013 13016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:49.841547 13000 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 11:55:49.841656 13000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "ce7151a62e2b414e8f4092cb5cbd6fb3"
format_stamp: "Formatted at 2026-08-11 11:55:49 on dist-test-slave-zkpd"
server_key: "41aae06ebf63227276e1a094e5d5c43f"
server_key_iv: "7c9821810f45c1c4dc9246efb8d2fa94"
server_key_version: "encryptionkey@0"
I20260811 11:55:49.841747 13000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:49.854259 13000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:49.857177 13000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:49.857362 13000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:49.857928 13000 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:55:49.858721 13000 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:49.858768 13000 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:49.858834 13000 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:49.858875 13000 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:49.867772 13000 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:39453
I20260811 11:55:49.867812 13138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:39453 every 8 connection(s)
I20260811 11:55:49.868681 13000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
I20260811 11:55:49.875730 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 13000
I20260811 11:55:49.875830 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance
I20260811 11:55:49.876158 6718 external_mini_cluster.cc:1428] Setting key 6b80ca44954908585ccb8abecfffee15
Aug 11 11:55:49 dist-test-slave-zkpd krb5kdc[12740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449349, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:49.882599 12831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:49.870493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:58001 (local address 127.6.143.190:44703)
0811 11:55:49.870622 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:49.870626 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:49.871442 (+ 816us) server_negotiation.cc:407] Connection header received
0811 11:55:49.872908 (+ 1466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:49.872912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:49.872972 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:49.873073 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:49.874225 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:49.874747 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:49.875942 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:49.876211 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:49.876329 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:49.878984 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:49.878987 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:49.878989 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:49.879012 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:49.880520 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:49.881050 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:49.881058 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:49.881059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:49.881133 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:49.881492 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:49.881496 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:49.881498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:49.881706 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:49.881813 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:49.882307 (+ 494us) server_negotiation.cc:299] Negotiation successful
0811 11:55:49.882431 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 11:55:49.883374 13142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:49.870847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:44703 (local address 127.6.143.130:58001)
0811 11:55:49.871296 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:49.871337 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:49.872686 (+ 1349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:49.873248 (+ 562us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:49.873255 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:49.873601 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:49.874077 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:49.874090 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:49.874873 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:49.874878 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:49.875768 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:49.875804 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:49.876427 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:49.876432 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:49.876638 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:49.877213 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:49.877231 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:49.878808 (+ 1577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:49.880647 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:49.880653 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:49.880668 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:49.880934 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:49.881229 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:49.881233 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:49.881235 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:49.881390 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:49.881828 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:49.881836 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:49.882072 (+ 236us) client_negotiation.cc:769] Sending connection context
0811 11:55:49.882258 (+ 186us) client_negotiation.cc:240] Negotiation successful
0811 11:55:49.882571 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":121,"threads_started":1}
I20260811 11:55:49.884538 13139 heartbeater.cc:347] Connected to a master server at 127.6.143.190:44703
I20260811 11:55:49.884796 13139 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:49.885419 13139 heartbeater.cc:511] Master 127.6.143.190:44703 requested a full tablet report, sending...
I20260811 11:55:49.886616 12791 ts_manager.cc:194] Registered new tserver with Master: ce7151a62e2b414e8f4092cb5cbd6fb3 (127.6.143.130:39453)
WARNING: no policy specified for kudu/127.6.143.131@KRBTEST.COM; defaulting to no policy
I20260811 11:55:49.887362 12791 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.130@KRBTEST.COM'} at 127.6.143.130:58001
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
I20260811 11:55:49.904501 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:0
--local_ip_for_outbound_sockets=127.6.143.131
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:44703
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:37599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:50.018678 13145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:50.018934 13145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:50.018976 13145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:50.022969 13145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:50.023028 13145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:50.023087 13145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:55:50.028421 13145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:37599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:44703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:50.029428 13145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:50.030447 13145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:50.040004 13155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:50.040000 13152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:50.040068 13153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:50.040359 13145 server_base.cc:1034] running on GCE node
I20260811 11:55:50.040788 13145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:50.041432 13145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:50.042618 13145 hybrid_clock.cc:648] HybridClock initialized: now 1786449350042602 us; error 26 us; skew 500 ppm
Aug 11 11:55:50 dist-test-slave-zkpd krb5kdc[12740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449350, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:50.045423 13145 init.cc:377] Logged in from keytab as kudu/127.6.143.131@KRBTEST.COM (short username kudu)
I20260811 11:55:50.046231 13145 webserver.cc:460] Webserver started at http://127.6.143.131:42479/ using document root <none> and password file <none>
I20260811 11:55:50.046861 13145 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:50.046952 13145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:50.047183 13145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:50.049214 13145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-2/data/instance:
uuid: "02da1f273cb9460d86d7aae02bfb1ec4"
format_stamp: "Formatted at 2026-08-11 11:55:50 on dist-test-slave-zkpd"
server_key: "2e6935c5f2b02804379934e8844ce5d2"
server_key_iv: "95c8f9550e8372256caef565715116f2"
server_key_version: "encryptionkey@0"
I20260811 11:55:50.049788 13145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance:
uuid: "02da1f273cb9460d86d7aae02bfb1ec4"
format_stamp: "Formatted at 2026-08-11 11:55:50 on dist-test-slave-zkpd"
server_key: "2e6935c5f2b02804379934e8844ce5d2"
server_key_iv: "95c8f9550e8372256caef565715116f2"
server_key_version: "encryptionkey@0"
I20260811 11:55:50.053362 13145 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 11:55:50.055750 13162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:50.057003 13145 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:55:50.057149 13145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "02da1f273cb9460d86d7aae02bfb1ec4"
format_stamp: "Formatted at 2026-08-11 11:55:50 on dist-test-slave-zkpd"
server_key: "2e6935c5f2b02804379934e8844ce5d2"
server_key_iv: "95c8f9550e8372256caef565715116f2"
server_key_version: "encryptionkey@0"
I20260811 11:55:50.057260 13145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:50.068290 13145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:50.071049 13145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:50.071245 13145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:50.071871 13145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:55:50.072779 13145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:50.072849 13145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:50.072929 13145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:50.072981 13145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:50.081627 13145 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:38521
I20260811 11:55:50.081699 13295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:38521 every 8 connection(s)
I20260811 11:55:50.082526 13145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
Aug 11 11:55:50 dist-test-slave-zkpd krb5kdc[12740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449350, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:55:50.092166 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 13145
I20260811 11:55:50.092290 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance
I20260811 11:55:50.092572 6718 external_mini_cluster.cc:1428] Setting key 04431fefd89a022e1db31ec2ae66cff8
I20260811 11:55:50.095860 12831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:50.084263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:53993 (local address 127.6.143.190:44703)
0811 11:55:50.084413 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:50.084418 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:50.085164 (+ 746us) server_negotiation.cc:407] Connection header received
0811 11:55:50.086466 (+ 1302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:50.086469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:50.086517 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:50.086591 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:50.087644 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:50.088230 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:50.089288 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:50.089501 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:50.089574 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:50.092079 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:50.092082 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:50.092084 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:50.092106 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:50.093885 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:50.094429 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:50.094433 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:50.094435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:50.094526 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:50.094954 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:50.094956 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:50.094958 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:50.095130 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:50.095208 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:50.095653 (+ 445us) server_negotiation.cc:299] Negotiation successful
0811 11:55:50.095744 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 11:55:50.096755 13299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:50.084606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:44703 (local address 127.6.143.131:53993)
0811 11:55:50.085022 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:50.085063 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:50.086273 (+ 1210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:50.086718 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:50.086725 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:50.087062 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:50.087522 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:50.087535 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:50.088339 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:50.088341 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:50.089184 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:50.089193 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:50.089624 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:50.089627 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:50.089771 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:50.090335 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:50.090351 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:50.091847 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:50.094021 (+ 2174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:50.094027 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:50.094041 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:50.094329 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:50.094616 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:50.094618 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:50.094619 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:50.094812 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:50.095223 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:50.095228 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:50.095450 (+ 222us) client_negotiation.cc:769] Sending connection context
0811 11:55:50.095604 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 11:55:50.095840 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":105,"threads_started":1}
I20260811 11:55:50.097853 13296 heartbeater.cc:347] Connected to a master server at 127.6.143.190:44703
I20260811 11:55:50.098062 13296 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:50.098495 13296 heartbeater.cc:511] Master 127.6.143.190:44703 requested a full tablet report, sending...
I20260811 11:55:50.099449 12791 ts_manager.cc:194] Registered new tserver with Master: 02da1f273cb9460d86d7aae02bfb1ec4 (127.6.143.131:38521)
I20260811 11:55:50.100028 12791 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.131@KRBTEST.COM'} at 127.6.143.131:53993
I20260811 11:55:50.106335 6718 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/test-admin.keytab.
Aug 11 11:55:50 dist-test-slave-zkpd krb5kdc[12740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:50.122416 6718 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 11:55:50.125173 13307 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:55:50.123744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49004 (local address 127.6.143.129:42505)
0811 11:55:50.124084 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:50.124100 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:50.124123 (+ 23us) server_negotiation.cc:407] Connection header received
0811 11:55:50.124175 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:50.124180 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:50.124367 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:50.124491 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:50.124986 (+ 495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49004: BlockingRecv error: recv got EOF from 127.0.0.1:49004 (error 108)
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":89,"threads_started":1}
W20260811 11:55:50.125172 6718 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/krb5cc' of type FILE
Aug 11 11:55:50 dist-test-slave-zkpd krb5kdc[12740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:50.126225 6718 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 11:55:50 dist-test-slave-zkpd krb5kdc[12740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:50.127180 6718 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 11:55:50 dist-test-slave-zkpd krb5kdc[12740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.129@KRBTEST.COM
I20260811 11:55:50.136979 13307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:50.128153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49014 (local address 127.6.143.129:42505)
0811 11:55:50.128280 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:50.128284 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:50.128385 (+ 101us) server_negotiation.cc:407] Connection header received
0811 11:55:50.128539 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:50.128542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:50.128586 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:50.128677 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:50.129216 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:50.130001 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:50.130847 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:50.131236 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:50.131350 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:50.133608 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:50.133623 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:50.133638 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:50.133661 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:50.135130 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:50.135609 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:50.135615 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:50.135616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:50.135692 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:50.136002 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:50.136006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:50.136008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:50.136250 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:50.136469 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:50.136694 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 11:55:50.136851 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 11:55:50.140775 13307 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:55:50.139706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49020 (local address 127.6.143.129:42505)
0811 11:55:50.139830 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:50.139834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:50.139991 (+ 157us) server_negotiation.cc:407] Connection header received
0811 11:55:50.140141 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:50.140144 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:50.140186 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:50.140266 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:50.140676 (+ 410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49020: BlockingRecv error: recv got EOF from 127.0.0.1:49020 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 11:55:50.141141 6718 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/krb5cc)
Aug 11 11:55:50 dist-test-slave-zkpd krb5kdc[12740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:50.142151 6718 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 11:55:50 dist-test-slave-zkpd krb5kdc[12740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:50.143020 6718 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 11:55:50 dist-test-slave-zkpd krb5kdc[12740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.129@KRBTEST.COM
I20260811 11:55:50.151551 13307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:50.143781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49034 (local address 127.6.143.129:42505)
0811 11:55:50.143934 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:50.143938 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:50.144038 (+ 100us) server_negotiation.cc:407] Connection header received
0811 11:55:50.144124 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:50.144126 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:50.144160 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:50.144224 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:50.144674 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:50.145180 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:50.146007 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:50.146216 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:50.146291 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:50.148532 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:50.148536 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:50.148538 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:50.148567 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:50.149961 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:50.150384 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:50.150388 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:50.150390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:50.150472 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:50.150750 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:50.150753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:50.150755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:50.150915 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:50.151039 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:50.151288 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 11:55:50.151437 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
W20260811 11:55:50.154167 13307 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:55:50.153163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49040 (local address 127.6.143.129:42505)
0811 11:55:50.153290 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:50.153294 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:50.153408 (+ 114us) server_negotiation.cc:407] Connection header received
0811 11:55:50.153541 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:50.153544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:50.153581 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:50.153670 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:50.154056 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49040: BlockingRecv error: recv got EOF from 127.0.0.1:49040 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 11:55:50.154328 6718 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786449319379472-6718-0/krb5kdc/krb5cc)
Aug 11 11:55:50 dist-test-slave-zkpd krb5kdc[12740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:50.155272 6718 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 11:55:50 dist-test-slave-zkpd krb5kdc[12740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:50.156176 6718 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 11:55:50 dist-test-slave-zkpd krb5kdc[12740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449350, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.129@KRBTEST.COM
I20260811 11:55:50.164085 13307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:50.156925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49054 (local address 127.6.143.129:42505)
0811 11:55:50.157052 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:50.157055 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:50.157085 (+ 30us) server_negotiation.cc:407] Connection header received
0811 11:55:50.157187 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:50.157189 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:50.157221 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:50.157282 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:50.157707 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:50.158165 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:50.158834 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:50.159022 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:50.159091 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:50.161184 (+ 2093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:50.161187 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:50.161189 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:50.161209 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:50.162537 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:50.162938 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:50.162942 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:50.162943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:50.163019 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:50.163300 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:50.163304 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:50.163305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:50.163463 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:50.163592 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:50.163825 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 11:55:50.163950 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 11:55:50.164938 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 12856
I20260811 11:55:50.173920 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 13000
I20260811 11:55:50.182360 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 13145
I20260811 11:55:50.188863 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 12754
2026-08-11T11:55:50Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (3538 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:55:50 dist-test-slave-zkpd krb5kdc[13322](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:55:50 dist-test-slave-zkpd krb5kdc[13322](info): set up 2 sockets
Aug 11 11:55:50 dist-test-slave-zkpd krb5kdc[13322](info): commencing operation
krb5kdc: starting...
W20260811 11:55:51.281554 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:55:51 dist-test-slave-zkpd krb5kdc[13322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:55:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:55:51Z Disabled control of system clock
WARNING: no policy specified for oryx/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.190.keytab.
Entry for principal oryx/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.190.keytab.
I20260811 11:55:51.402153 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:46619
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.190.keytab
--principal=oryx/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:45207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:46619
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:51.516644 13337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:51.516906 13337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:51.516983 13337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:51.520865 13337 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:55:51.520938 13337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:51.520978 13337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:51.521035 13337 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:55:51.521087 13337 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:55:51.526253 13337 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:45207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:46619
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.190.keytab
--principal=oryx/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:46619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:51.527213 13337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:51.528229 13337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:51.536336 13343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:51.536350 13345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:51.536362 13342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:51.536412 13337 server_base.cc:1034] running on GCE node
I20260811 11:55:51.537195 13337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:51.538281 13337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:51.539454 13337 hybrid_clock.cc:648] HybridClock initialized: now 1786449351539424 us; error 40 us; skew 500 ppm
Aug 11 11:55:51 dist-test-slave-zkpd krb5kdc[13322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449351, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:51.545490 13337 init.cc:377] Logged in from keytab as oryx/127.6.143.190@KRBTEST.COM (short username oryx)
I20260811 11:55:51.546307 13337 webserver.cc:460] Webserver started at http://127.6.143.190:46415/ using document root <none> and password file <none>
I20260811 11:55:51.546861 13337 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:51.546945 13337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:51.547164 13337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:51.549256 13337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "8fedd24da8fe4ac59ffba44f177b1009"
format_stamp: "Formatted at 2026-08-11 11:55:51 on dist-test-slave-zkpd"
server_key: "b147449361da8d4756ea920f9bbc2a0f"
server_key_iv: "4e3a448e0a114ec06f2272073839a5c6"
server_key_version: "encryptionkey@0"
I20260811 11:55:51.549819 13337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "8fedd24da8fe4ac59ffba44f177b1009"
format_stamp: "Formatted at 2026-08-11 11:55:51 on dist-test-slave-zkpd"
server_key: "b147449361da8d4756ea920f9bbc2a0f"
server_key_iv: "4e3a448e0a114ec06f2272073839a5c6"
server_key_version: "encryptionkey@0"
I20260811 11:55:51.553350 13337 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 11:55:51.555644 13352 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:51.556910 13337 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 11:55:51.557057 13337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "8fedd24da8fe4ac59ffba44f177b1009"
format_stamp: "Formatted at 2026-08-11 11:55:51 on dist-test-slave-zkpd"
server_key: "b147449361da8d4756ea920f9bbc2a0f"
server_key_iv: "4e3a448e0a114ec06f2272073839a5c6"
server_key_version: "encryptionkey@0"
I20260811 11:55:51.557156 13337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:51.580559 13337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:51.583114 13337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:51.583300 13337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:51.590071 13337 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:46619
I20260811 11:55:51.590102 13409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:46619 every 8 connection(s)
I20260811 11:55:51.590973 13337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:55:51.593668 13410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:51.598589 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 13337
I20260811 11:55:51.598737 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:55:51.599013 6718 external_mini_cluster.cc:1428] Setting key 9b6d6eb94bf0a76d7cc0b825b1960025
I20260811 11:55:51.599090 13410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009: Bootstrap starting.
I20260811 11:55:51.601735 13410 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:51.602615 13410 log.cc:824] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:51.604496 13410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009: No bootstrap required, opened a new log
Aug 11 11:55:51 dist-test-slave-zkpd krb5kdc[13322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM
I20260811 11:55:51.608096 13410 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } }
I20260811 11:55:51.608341 13410 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:51.608425 13410 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fedd24da8fe4ac59ffba44f177b1009, State: Initialized, Role: FOLLOWER
I20260811 11:55:51.608878 13410 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } }
I20260811 11:55:51.609026 13410 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:55:51.609097 13410 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:55:51.609191 13410 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:51.609988 13410 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } }
I20260811 11:55:51.610316 13410 leader_election.cc:302] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8fedd24da8fe4ac59ffba44f177b1009; no voters:
I20260811 11:55:51.610644 13410 leader_election.cc:288] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:55:51.610704 13416 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:51.610949 13416 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [term 1 LEADER]: Becoming Leader. State: Replica: 8fedd24da8fe4ac59ffba44f177b1009, State: Running, Role: LEADER
I20260811 11:55:51.611281 13416 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } }
I20260811 11:55:51.611963 13410 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:55:51.612502 13416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8fedd24da8fe4ac59ffba44f177b1009" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } }
I20260811 11:55:51.612617 13416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:51.613111 13417 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fedd24da8fe4ac59ffba44f177b1009. Latest consensus state: current_term: 1 leader_uuid: "8fedd24da8fe4ac59ffba44f177b1009" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } }
I20260811 11:55:51.613212 13417 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:51.613613 13421 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:55:51.614055 13414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:51.600340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51130 (local address 127.6.143.190:46619)
0811 11:55:51.600764 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:51.600782 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:51.600823 (+ 41us) server_negotiation.cc:407] Connection header received
0811 11:55:51.601623 (+ 800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:51.601650 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:51.602094 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:51.602451 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:51.603147 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:51.604171 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:51.605072 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:51.605523 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:51.605690 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:51.608392 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:51.608398 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:51.608414 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:51.608451 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:51.610301 (+ 1850us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:51.610821 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:51.610828 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:51.610833 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:51.610981 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:51.611291 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:51.611294 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:51.611295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:51.611763 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:51.611944 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:51.612397 (+ 453us) server_negotiation.cc:299] Negotiation successful
0811 11:55:51.612867 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":115,"threads_started":1}
I20260811 11:55:51.616658 13421 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:55:51.622103 13421 catalog_manager.cc:1347] Generated new cluster ID: e3111c441033471b8db261e6829e1e9e
I20260811 11:55:51.622169 13421 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:55:51.646287 13421 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:55:51.647643 13421 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:55:51.655939 13421 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009: Generated new TSK 0
I20260811 11:55:51.656728 13421 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.6.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.129.keytab.
Entry for principal oryx/127.6.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.129.keytab.
I20260811 11:55:51.690714 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:0
--local_ip_for_outbound_sockets=127.6.143.129
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:46619
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.129.keytab
--principal=oryx/127.6.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:45207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:51.801538 13438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:51.801852 13438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:51.801934 13438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:51.805974 13438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:51.806073 13438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:51.806171 13438 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:55:51.811637 13438 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:45207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.129.keytab
--principal=oryx/127.6.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:46619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:51.812847 13438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:51.814010 13438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:51.823838 13444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:51.824004 13438 server_base.cc:1034] running on GCE node
W20260811 11:55:51.823838 13446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:51.823948 13443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:51.824609 13438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:51.825311 13438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:51.826507 13438 hybrid_clock.cc:648] HybridClock initialized: now 1786449351826492 us; error 29 us; skew 500 ppm
Aug 11 11:55:51 dist-test-slave-zkpd krb5kdc[13322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449351, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:51.829331 13438 init.cc:377] Logged in from keytab as oryx/127.6.143.129@KRBTEST.COM (short username oryx)
I20260811 11:55:51.830224 13438 webserver.cc:460] Webserver started at http://127.6.143.129:35417/ using document root <none> and password file <none>
I20260811 11:55:51.830767 13438 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:51.830825 13438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:51.831000 13438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:51.833107 13438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/data/instance:
uuid: "b063e255083e4cdda907e958091e2606"
format_stamp: "Formatted at 2026-08-11 11:55:51 on dist-test-slave-zkpd"
server_key: "67d15f2befefda95e9b5a829cfbeb17e"
server_key_iv: "81d26e395458042eb718feb7c55a47c8"
server_key_version: "encryptionkey@0"
I20260811 11:55:51.833639 13438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance:
uuid: "b063e255083e4cdda907e958091e2606"
format_stamp: "Formatted at 2026-08-11 11:55:51 on dist-test-slave-zkpd"
server_key: "67d15f2befefda95e9b5a829cfbeb17e"
server_key_iv: "81d26e395458042eb718feb7c55a47c8"
server_key_version: "encryptionkey@0"
I20260811 11:55:51.837744 13438 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 11:55:51.840179 13453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:51.841516 13438 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:55:51.841663 13438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "b063e255083e4cdda907e958091e2606"
format_stamp: "Formatted at 2026-08-11 11:55:51 on dist-test-slave-zkpd"
server_key: "67d15f2befefda95e9b5a829cfbeb17e"
server_key_iv: "81d26e395458042eb718feb7c55a47c8"
server_key_version: "encryptionkey@0"
I20260811 11:55:51.841785 13438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:51.858928 13438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:51.862154 13438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:51.862360 13438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:51.862957 13438 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:55:51.863816 13438 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:51.863866 13438 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:51.863967 13438 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:51.864010 13438 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:51.872996 13438 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:39527
I20260811 11:55:51.873025 13576 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:39527 every 8 connection(s)
I20260811 11:55:51.873940 13438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:55:51.877255 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 13438
I20260811 11:55:51.877411 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance
I20260811 11:55:51.877786 6718 external_mini_cluster.cc:1428] Setting key 4dfb7501c5c5f0bfc39f8203e5949b54
Aug 11 11:55:51 dist-test-slave-zkpd krb5kdc[13322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449351, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.6.143.130@KRBTEST.COM; defaulting to no policy
I20260811 11:55:51.889523 13414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:51.875780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:54939 (local address 127.6.143.190:46619)
0811 11:55:51.875968 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:51.875973 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:51.876866 (+ 893us) server_negotiation.cc:407] Connection header received
0811 11:55:51.878507 (+ 1641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:51.878511 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:51.878568 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:51.878657 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:51.880136 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:51.880663 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:51.882115 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:51.882355 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:51.882473 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:51.885293 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:51.885299 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:51.885305 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:51.885343 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:51.887115 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:51.887669 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:51.887672 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:51.887674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:51.887773 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:51.888231 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:51.888237 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:51.888242 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:51.888540 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:51.888659 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:51.889264 (+ 605us) server_negotiation.cc:299] Negotiation successful
0811 11:55:51.889388 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":66}
I20260811 11:55:51.890367 13579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:51.876236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.6.143.129:54939)
0811 11:55:51.876707 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:51.876752 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:51.878282 (+ 1530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:51.878858 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:51.878866 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:51.879266 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:51.879984 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:51.880007 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:51.880794 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:51.880801 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:51.881975 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:51.881985 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:51.882524 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:51.882528 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:51.882699 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:51.883311 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:51.883328 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:51.885098 (+ 1770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:51.887243 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:51.887249 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:51.887263 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:51.887540 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:51.887876 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:51.887878 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:51.887880 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:51.888105 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:51.888653 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:51.888658 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:51.888958 (+ 300us) client_negotiation.cc:769] Sending connection context
0811 11:55:51.889152 (+ 194us) client_negotiation.cc:240] Negotiation successful
0811 11:55:51.889436 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":104,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.143.130@KRBTEST.COM" created.
I20260811 11:55:51.891846 13577 heartbeater.cc:347] Connected to a master server at 127.6.143.190:46619
I20260811 11:55:51.892199 13577 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:51.892850 13577 heartbeater.cc:511] Master 127.6.143.190:46619 requested a full tablet report, sending...
I20260811 11:55:51.894572 13371 ts_manager.cc:194] Registered new tserver with Master: b063e255083e4cdda907e958091e2606 (127.6.143.129:39527)
I20260811 11:55:51.896245 13371 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.143.129@KRBTEST.COM'} at 127.6.143.129:54939
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.130.keytab.
Entry for principal oryx/127.6.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.130.keytab.
I20260811 11:55:51.910429 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:0
--local_ip_for_outbound_sockets=127.6.143.130
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:46619
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.130.keytab
--principal=oryx/127.6.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:45207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:52.023809 13582 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:52.024143 13582 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:52.024228 13582 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:52.028120 13582 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:52.028201 13582 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:52.028288 13582 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:55:52.033339 13582 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:45207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.130.keytab
--principal=oryx/127.6.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:46619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13582
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:52.034327 13582 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:52.035311 13582 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:52.044683 13590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:52.044690 13592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:52.044677 13588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:52.045007 13582 server_base.cc:1034] running on GCE node
I20260811 11:55:52.045439 13582 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:52.046080 13582 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:52.047240 13582 hybrid_clock.cc:648] HybridClock initialized: now 1786449352047203 us; error 45 us; skew 500 ppm
Aug 11 11:55:52 dist-test-slave-zkpd krb5kdc[13322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449352, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:52.049897 13582 init.cc:377] Logged in from keytab as oryx/127.6.143.130@KRBTEST.COM (short username oryx)
I20260811 11:55:52.050710 13582 webserver.cc:460] Webserver started at http://127.6.143.130:39337/ using document root <none> and password file <none>
I20260811 11:55:52.051206 13582 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:52.051262 13582 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:52.051426 13582 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:52.053377 13582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/data/instance:
uuid: "0e664a94aff147c5aea4d0c580f610fe"
format_stamp: "Formatted at 2026-08-11 11:55:52 on dist-test-slave-zkpd"
server_key: "d0ae82e29b01275185d3374cff2f3c5d"
server_key_iv: "83adbed30053a3e7bb8c8dd4427a2854"
server_key_version: "encryptionkey@0"
I20260811 11:55:52.053891 13582 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance:
uuid: "0e664a94aff147c5aea4d0c580f610fe"
format_stamp: "Formatted at 2026-08-11 11:55:52 on dist-test-slave-zkpd"
server_key: "d0ae82e29b01275185d3374cff2f3c5d"
server_key_iv: "83adbed30053a3e7bb8c8dd4427a2854"
server_key_version: "encryptionkey@0"
I20260811 11:55:52.057404 13582 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 11:55:52.059664 13599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:52.060899 13582 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 11:55:52.061031 13582 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "0e664a94aff147c5aea4d0c580f610fe"
format_stamp: "Formatted at 2026-08-11 11:55:52 on dist-test-slave-zkpd"
server_key: "d0ae82e29b01275185d3374cff2f3c5d"
server_key_iv: "83adbed30053a3e7bb8c8dd4427a2854"
server_key_version: "encryptionkey@0"
I20260811 11:55:52.061137 13582 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:52.078590 13582 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:52.081240 13582 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:52.081449 13582 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:52.082019 13582 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:55:52.082918 13582 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:52.082989 13582 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:52.083060 13582 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:52.083101 13582 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:52.092056 13582 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:42017
I20260811 11:55:52.092083 13720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:42017 every 8 connection(s)
I20260811 11:55:52.092895 13582 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
I20260811 11:55:52.098294 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 13582
I20260811 11:55:52.098439 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance
I20260811 11:55:52.098724 6718 external_mini_cluster.cc:1428] Setting key fa84a8c8b12b0d7baff91d66d5051677
Aug 11 11:55:52 dist-test-slave-zkpd krb5kdc[13322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449352, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM
I20260811 11:55:52.106446 13414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.094646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:41153 (local address 127.6.143.190:46619)
0811 11:55:52.094790 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:52.094794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:52.095587 (+ 793us) server_negotiation.cc:407] Connection header received
0811 11:55:52.096815 (+ 1228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:52.096818 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:52.096871 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:52.096946 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:52.098087 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.098699 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.099952 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.100265 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.100380 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.102797 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:52.102800 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:52.102802 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:52.102824 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:52.104402 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.104948 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.104952 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.104954 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.105037 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.105375 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.105381 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.105385 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.105570 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:52.105684 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:52.106158 (+ 474us) server_negotiation.cc:299] Negotiation successful
0811 11:55:52.106279 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 11:55:52.107203 13724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.095003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.6.143.130:41153)
0811 11:55:52.095442 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:52.095483 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:52.096620 (+ 1137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:52.097135 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:52.097142 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:52.097481 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:52.097943 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.097956 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.098830 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.098834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.099794 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.099805 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.100410 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.100413 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.100580 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.101163 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:52.101179 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:52.102631 (+ 1452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:52.104537 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.104543 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.104557 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.104824 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.105132 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.105136 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.105138 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.105276 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.105685 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:52.105692 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:52.105925 (+ 233us) client_negotiation.cc:769] Sending connection context
0811 11:55:52.106110 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 11:55:52.106423 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":114,"threads_started":1}
I20260811 11:55:52.108397 13721 heartbeater.cc:347] Connected to a master server at 127.6.143.190:46619
I20260811 11:55:52.108673 13721 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for oryx/127.6.143.131@KRBTEST.COM; defaulting to no policy
I20260811 11:55:52.109239 13721 heartbeater.cc:511] Master 127.6.143.190:46619 requested a full tablet report, sending...
I20260811 11:55:52.110373 13371 ts_manager.cc:194] Registered new tserver with Master: 0e664a94aff147c5aea4d0c580f610fe (127.6.143.130:42017)
I20260811 11:55:52.110911 13371 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.143.130@KRBTEST.COM'} at 127.6.143.130:41153
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.131.keytab.
Entry for principal oryx/127.6.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.131.keytab.
I20260811 11:55:52.127127 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:0
--local_ip_for_outbound_sockets=127.6.143.131
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:46619
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.131.keytab
--principal=oryx/127.6.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:45207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:52.238484 13727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:52.238715 13727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:52.238754 13727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:52.242823 13727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:52.242878 13727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:52.242938 13727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:55:52.248009 13727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:45207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.131.keytab
--principal=oryx/127.6.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:46619
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:52.248878 13727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:52.249792 13727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:52.259099 13735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:52.259091 13732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:52.259091 13733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:52.259323 13727 server_base.cc:1034] running on GCE node
I20260811 11:55:52.259716 13727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:52.260437 13727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:52.261633 13727 hybrid_clock.cc:648] HybridClock initialized: now 1786449352261612 us; error 32 us; skew 500 ppm
Aug 11 11:55:52 dist-test-slave-zkpd krb5kdc[13322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449352, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:52.264328 13727 init.cc:377] Logged in from keytab as oryx/127.6.143.131@KRBTEST.COM (short username oryx)
I20260811 11:55:52.265194 13727 webserver.cc:460] Webserver started at http://127.6.143.131:43327/ using document root <none> and password file <none>
I20260811 11:55:52.265730 13727 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:52.265791 13727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:52.266013 13727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:52.268023 13727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/data/instance:
uuid: "d8af8d74ff8b451e80081f46f94504a2"
format_stamp: "Formatted at 2026-08-11 11:55:52 on dist-test-slave-zkpd"
server_key: "e55eb35271deefff0b9fa4949150ccbe"
server_key_iv: "5aead28516a814495887373f58043f42"
server_key_version: "encryptionkey@0"
I20260811 11:55:52.268576 13727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance:
uuid: "d8af8d74ff8b451e80081f46f94504a2"
format_stamp: "Formatted at 2026-08-11 11:55:52 on dist-test-slave-zkpd"
server_key: "e55eb35271deefff0b9fa4949150ccbe"
server_key_iv: "5aead28516a814495887373f58043f42"
server_key_version: "encryptionkey@0"
I20260811 11:55:52.272228 13727 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 11:55:52.274557 13742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:52.275743 13727 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:52.275836 13727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "d8af8d74ff8b451e80081f46f94504a2"
format_stamp: "Formatted at 2026-08-11 11:55:52 on dist-test-slave-zkpd"
server_key: "e55eb35271deefff0b9fa4949150ccbe"
server_key_iv: "5aead28516a814495887373f58043f42"
server_key_version: "encryptionkey@0"
I20260811 11:55:52.275975 13727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:52.298082 13727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:52.301869 13727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:52.302016 13727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:52.302543 13727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:55:52.303403 13727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:52.303452 13727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:52.303489 13727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:52.303509 13727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:52.311856 13727 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:42641
I20260811 11:55:52.311916 13864 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:42641 every 8 connection(s)
I20260811 11:55:52.312835 13727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:55:52.313054 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 13727
I20260811 11:55:52.313215 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance
I20260811 11:55:52.313575 6718 external_mini_cluster.cc:1428] Setting key cf7499785bf4c5d521b58ebebb7ae694
Aug 11 11:55:52 dist-test-slave-zkpd krb5kdc[13322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449352, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM
I20260811 11:55:52.325776 13414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.314829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:41729 (local address 127.6.143.190:46619)
0811 11:55:52.314985 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:52.314990 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:52.316061 (+ 1071us) server_negotiation.cc:407] Connection header received
0811 11:55:52.317340 (+ 1279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:52.317343 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:52.317389 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:52.317457 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:52.318510 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.319030 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.320172 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.320389 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.320470 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.322710 (+ 2240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:52.322713 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:52.322715 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:52.322738 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:52.324060 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.324519 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.324521 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.324523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.324597 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.324893 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.324895 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.324896 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.325066 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:52.325142 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:52.325567 (+ 425us) server_negotiation.cc:299] Negotiation successful
0811 11:55:52.325661 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 11:55:52.326612 13867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.315364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.6.143.131:41729)
0811 11:55:52.315851 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:52.315959 (+ 108us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:52.317149 (+ 1190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:52.317593 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:52.317599 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:52.317941 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:52.318388 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.318400 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.319138 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.319141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.320062 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.320072 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.320515 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.320517 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.320648 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.321129 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:52.321144 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:52.322501 (+ 1357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:52.324164 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.324168 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.324179 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.324433 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.324676 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.324678 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.324679 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.324804 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.325153 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:52.325158 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:52.325360 (+ 202us) client_negotiation.cc:769] Sending connection context
0811 11:55:52.325518 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 11:55:52.325745 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":105,"threads_started":1}
I20260811 11:55:52.327729 13865 heartbeater.cc:347] Connected to a master server at 127.6.143.190:46619
I20260811 11:55:52.328012 13865 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:52.328444 13865 heartbeater.cc:511] Master 127.6.143.190:46619 requested a full tablet report, sending...
I20260811 11:55:52.329432 13371 ts_manager.cc:194] Registered new tserver with Master: d8af8d74ff8b451e80081f46f94504a2 (127.6.143.131:42641)
I20260811 11:55:52.329998 13371 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.143.131@KRBTEST.COM'} at 127.6.143.131:41729
I20260811 11:55:52.339548 6718 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.6.143.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.143.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.143.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.189.keytab.
Entry for principal oryx/127.6.143.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.189.keytab.
I20260811 11:55:52.366972 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.189:43751
--webserver_interface=127.6.143.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.189.keytab
--principal=oryx/127.6.143.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:45207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:46619,127.6.143.189:43751
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:52.480234 13871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:52.480486 13871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:52.480551 13871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:52.484733 13871 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:55:52.484782 13871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:52.484802 13871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:52.484840 13871 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:55:52.484877 13871 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:55:52.489897 13871 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:45207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:46619,127.6.143.189:43751
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.189.keytab
--principal=oryx/127.6.143.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.189:43751
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.143.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
W20260811 11:55:52.490057 13871 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.6.143.190:46619,127.6.143.189:43751'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 11:55:52.490906 13871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:52.491971 13871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:52.500239 13876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:52.500264 13879 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:52.500537 13871 server_base.cc:1034] running on GCE node
W20260811 11:55:52.500705 13877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:52.501291 13871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:52.502333 13871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:52.503489 13871 hybrid_clock.cc:648] HybridClock initialized: now 1786449352503475 us; error 24 us; skew 500 ppm
Aug 11 11:55:52 dist-test-slave-zkpd krb5kdc[13322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449352, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:52.506148 13871 init.cc:377] Logged in from keytab as oryx/127.6.143.189@KRBTEST.COM (short username oryx)
I20260811 11:55:52.506968 13871 webserver.cc:460] Webserver started at http://127.6.143.189:38949/ using document root <none> and password file <none>
I20260811 11:55:52.507505 13871 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:52.507566 13871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:52.507805 13871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:52.509799 13871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-1/data/instance:
uuid: "79d3c932590e456c86fee8a8faecbec3"
format_stamp: "Formatted at 2026-08-11 11:55:52 on dist-test-slave-zkpd"
server_key: "ad96a12fa0c6bd049a8bb15dbcef2e92"
server_key_iv: "da00dde73a81b940e8cedf6c9949e8a4"
server_key_version: "encryptionkey@0"
I20260811 11:55:52.510373 13871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-1/wal/instance:
uuid: "79d3c932590e456c86fee8a8faecbec3"
format_stamp: "Formatted at 2026-08-11 11:55:52 on dist-test-slave-zkpd"
server_key: "ad96a12fa0c6bd049a8bb15dbcef2e92"
server_key_iv: "da00dde73a81b940e8cedf6c9949e8a4"
server_key_version: "encryptionkey@0"
I20260811 11:55:52.514309 13871 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260811 11:55:52.516594 13886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:52.517750 13871 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:52.517874 13871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-1/wal
uuid: "79d3c932590e456c86fee8a8faecbec3"
format_stamp: "Formatted at 2026-08-11 11:55:52 on dist-test-slave-zkpd"
server_key: "ad96a12fa0c6bd049a8bb15dbcef2e92"
server_key_iv: "da00dde73a81b940e8cedf6c9949e8a4"
server_key_version: "encryptionkey@0"
I20260811 11:55:52.517974 13871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:52.528087 13871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:52.530589 13871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:52.530777 13871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:52.537737 13871 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.189:43751
I20260811 11:55:52.538033 13942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.189:43751 every 8 connection(s)
I20260811 11:55:52.538643 13871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-1/data/info.pb
I20260811 11:55:52.541381 13943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:52.542934 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 13871
I20260811 11:55:52.545343 13943 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 11:55:52 dist-test-slave-zkpd krb5kdc[13322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.143.189@KRBTEST.COM
Aug 11 11:55:52 dist-test-slave-zkpd krb5kdc[13322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449352, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.189@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM
I20260811 11:55:52.556883 13414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.547352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51132 (local address 127.6.143.190:46619)
0811 11:55:52.547518 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:52.547522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:52.547734 (+ 212us) server_negotiation.cc:407] Connection header received
0811 11:55:52.547937 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:52.547940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:52.547996 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:52.548084 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:52.548852 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.549400 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.550491 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.550721 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.550829 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.553473 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:52.553477 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:52.553482 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:52.553516 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:52.555040 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.555489 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.555493 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.555495 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.555582 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.555861 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.555866 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.555870 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.556127 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:52.556238 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:52.556594 (+ 356us) server_negotiation.cc:299] Negotiation successful
0811 11:55:52.556708 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 11:55:52.557610 13947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.547263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.0.0.1:51132)
0811 11:55:52.547613 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:52.547649 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:52.547789 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:52.548136 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:52.548142 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:52.548560 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:52.548716 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.548724 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.549531 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.549534 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.550356 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.550373 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.550878 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.550882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.550989 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.551489 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:52.551505 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:52.553305 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:52.555143 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.555153 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.555162 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.555384 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.555675 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.555679 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.555681 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.555767 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.556237 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:52.556244 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:52.556377 (+ 133us) client_negotiation.cc:769] Sending connection context
0811 11:55:52.556524 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 11:55:52.556834 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":54,"thread_start_us":87,"threads_started":1}
I20260811 11:55:52.557613 13946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.544330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38368 (local address 127.6.143.189:43751)
0811 11:55:52.544745 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:52.544760 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:52.544794 (+ 34us) server_negotiation.cc:407] Connection header received
0811 11:55:52.545391 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:52.545408 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:52.545816 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:52.546102 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:52.547009 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.548360 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.549338 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.549874 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.549994 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.552817 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:52.552826 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:52.552842 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:52.552876 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:52.554758 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.555192 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.555203 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.555204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.555309 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.555601 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.555605 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.555606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.556050 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:52.556253 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:52.556497 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 11:55:52.556778 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":114,"threads_started":1}
I20260811 11:55:52.558972 13943 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 11:55:52 dist-test-slave-zkpd krb5kdc[13322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449352, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.189@KRBTEST.COM for oryx/127.6.143.189@KRBTEST.COM
I20260811 11:55:52.566501 13947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.559516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:43751 (local address 127.0.0.1:38374)
0811 11:55:52.559663 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:52.559676 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:52.559749 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:52.560016 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:52.560018 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:52.560141 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:52.560223 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.560228 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.561272 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.561274 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.561798 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.561806 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.562199 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.562201 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.562269 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.562626 (+ 357us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:52.562636 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:52.563737 (+ 1101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:52.565290 (+ 1553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.565293 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.565294 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.565482 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.565745 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.565748 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.565749 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.565821 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.566140 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:52.566142 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:52.566211 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 11:55:52.566289 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 11:55:52.566378 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260811 11:55:52.566552 13946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.559591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38374 (local address 127.6.143.189:43751)
0811 11:55:52.559688 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:52.559693 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:52.559709 (+ 16us) server_negotiation.cc:407] Connection header received
0811 11:55:52.559843 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:52.559845 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:52.559879 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:52.559997 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:52.560316 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.561173 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.561891 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.562107 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.562185 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.563865 (+ 1680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:52.563870 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:52.563876 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:52.563937 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:52.565193 (+ 1256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.565564 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.565567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.565586 (+ 19us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.565678 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.565912 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.565914 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.565915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.566060 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:52.566132 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:52.566343 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 11:55:52.566424 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 11:55:52.569235 13943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3: Bootstrap starting.
I20260811 11:55:52.571419 13943 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:52.572206 13943 log.cc:824] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:52.573968 13943 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3: No bootstrap required, opened a new log
I20260811 11:55:52.576974 13943 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } }
I20260811 11:55:52.577191 13943 raft_consensus.cc:374] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:52.577270 13943 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79d3c932590e456c86fee8a8faecbec3, State: Initialized, Role: FOLLOWER
I20260811 11:55:52.577689 13943 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } }
I20260811 11:55:52.578133 13949 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } } }
I20260811 11:55:52.578239 13949 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:55:52.578842 13943 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 11:55:52.584403 13960 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 11:55:52.584483 13960 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 11:55:52.585631 13871 master_runner.cc:428] Detected that this master 79d3c932590e456c86fee8a8faecbec3 is joining an existing cluster
I20260811 11:55:52.585678 13871 master_runner.cc:432] Initiating AddMaster RPC to add 127.6.143.189:43751
I20260811 11:55:52.597287 13414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.587685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51146 (local address 127.6.143.190:46619)
0811 11:55:52.587788 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:52.587793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:52.588122 (+ 329us) server_negotiation.cc:407] Connection header received
0811 11:55:52.588305 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:52.588308 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:52.588355 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:52.588427 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:52.589136 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.589923 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.591052 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.591405 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.591470 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.593515 (+ 2045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:52.593520 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:52.593525 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:52.593555 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:52.595345 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.595953 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.595956 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.595957 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.596036 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.596379 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.596383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.596384 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.596575 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:52.596651 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:52.597023 (+ 372us) server_negotiation.cc:299] Negotiation successful
0811 11:55:52.597137 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 11:55:52.597287 13946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.587684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38386 (local address 127.6.143.189:43751)
0811 11:55:52.587788 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:52.587793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:52.588263 (+ 470us) server_negotiation.cc:407] Connection header received
0811 11:55:52.588404 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:52.588409 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:52.588446 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:52.588536 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:52.589071 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.589813 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.591040 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.591346 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.591416 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.593265 (+ 1849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:52.593271 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:52.593273 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:52.593299 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:52.595437 (+ 2138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.595951 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.595954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.595956 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.596036 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.596359 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.596363 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.596365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.596564 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:52.596651 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:52.597023 (+ 372us) server_negotiation.cc:299] Negotiation successful
0811 11:55:52.597137 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 11:55:52.597841 13968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.587879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:43751 (local address 127.0.0.1:38386)
0811 11:55:52.588171 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:52.588183 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:52.588297 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:52.588548 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:52.588550 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:52.588844 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:52.588955 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.588961 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.589938 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.589940 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.590916 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.590927 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.591587 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.591591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.591704 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.592518 (+ 814us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:52.592536 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:52.593131 (+ 595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:52.595543 (+ 2412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.595545 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.595547 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.595830 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.596143 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.596146 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.596148 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.596261 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.596699 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:52.596702 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:52.596832 (+ 130us) client_negotiation.cc:769] Sending connection context
0811 11:55:52.597480 (+ 648us) client_negotiation.cc:240] Negotiation successful
0811 11:55:52.597644 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"mutex_wait_us":120,"spinlock_wait_cycles":8320,"thread_start_us":107,"threads_started":1}
I20260811 11:55:52.597877 13967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.587631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.0.0.1:51146)
0811 11:55:52.588037 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:52.588052 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:52.588195 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:52.588590 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:52.588593 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:52.588894 (+ 301us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:52.589028 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.589036 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.590043 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.590046 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.590919 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.590927 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.591604 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.591607 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.591726 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.592873 (+ 1147us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:52.592887 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:52.593339 (+ 452us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:52.595482 (+ 2143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.595485 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.595488 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.595813 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.596143 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.596146 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.596148 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.596269 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.596700 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:52.596703 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:52.596833 (+ 130us) client_negotiation.cc:769] Sending connection context
0811 11:55:52.597489 (+ 656us) client_negotiation.cc:240] Negotiation successful
0811 11:55:52.597720 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":185,"spinlock_wait_cycles":20992,"thread_start_us":90,"threads_started":1}
Aug 11 11:55:52 dist-test-slave-zkpd krb5kdc[13322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449351, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.190@KRBTEST.COM for oryx/127.6.143.189@KRBTEST.COM
I20260811 11:55:52.613173 13946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.602258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38398 (local address 127.6.143.189:43751)
0811 11:55:52.602386 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:52.602391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:52.604253 (+ 1862us) server_negotiation.cc:407] Connection header received
0811 11:55:52.604463 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:52.604466 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:52.604519 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:52.604604 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:52.605197 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.605665 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.606523 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.606736 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.606843 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.609179 (+ 2336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:52.609182 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:52.609185 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:52.609215 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:52.610775 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.611331 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.611337 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.611341 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.611431 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.611828 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.611834 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.611839 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.612131 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:52.612235 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:52.612811 (+ 576us) server_negotiation.cc:299] Negotiation successful
0811 11:55:52.612951 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 11:55:52.613915 13969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.602232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:43751 (local address 127.0.0.1:38398)
0811 11:55:52.604139 (+ 1907us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:52.604173 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:52.604322 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:52.604640 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:52.604646 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:52.604985 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:52.605081 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.605087 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.605774 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.605779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.606399 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.606408 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.606846 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.606850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.606968 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.607472 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:52.607484 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:52.609045 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:52.610906 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.610916 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.610922 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.611217 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.611620 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.611624 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.611626 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.611727 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.612299 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:52.612308 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:52.612486 (+ 178us) client_negotiation.cc:769] Sending connection context
0811 11:55:52.613443 (+ 957us) client_negotiation.cc:240] Negotiation successful
0811 11:55:52.613732 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1767,"thread_start_us":116,"threads_started":1}
I20260811 11:55:52.615258 13371 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.6.143.189:43751: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "79d3c932590e456c86fee8a8faecbec3"
member_type: NON_VOTER
last_known_addr {
host: "127.6.143.189"
port: 43751
}
attrs {
promote: true
}
}
dest_uuid: "8fedd24da8fe4ac59ffba44f177b1009"
cas_config_opid_index: -1
I20260811 11:55:52.615978 13371 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: NON_VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: true } }
I20260811 11:55:52.617895 13975 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 79d3c932590e456c86fee8a8faecbec3 (127.6.143.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: NON_VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: true } } }
I20260811 11:55:52.618573 13916 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:52.619590 13975 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.6.143.189:43751
I20260811 11:55:52.619947 13975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8fedd24da8fe4ac59ffba44f177b1009" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: NON_VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: true } } }
I20260811 11:55:52.620047 13975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:52.620044 13916 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [term 1 FOLLOWER]: Refusing update from remote peer 8fedd24da8fe4ac59ffba44f177b1009: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 11:55:52.620071 13871 master.cc:537] Master@127.6.143.189:43751 shutting down...
I20260811 11:55:52.620674 13976 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: NON_VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 11:55:52.622978 13356 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260811 11:55:52.624189 13356 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 -> Peer 79d3c932590e456c86fee8a8faecbec3 (127.6.143.189:43751): Couldn't send request to peer 79d3c932590e456c86fee8a8faecbec3. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 11:55:52.625259 13871 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [term 1 FOLLOWER]: Raft consensus shutting down.
WARNING: no policy specified for oryx/127.6.143.188@KRBTEST.COM; defaulting to no policy
I20260811 11:55:52.625572 13871 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:52.625700 13871 tablet_replica.cc:330] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.143.188@KRBTEST.COM" created.
I20260811 11:55:52.631654 13871 master.cc:552] Master@127.6.143.189:43751 shutdown complete.
I20260811 11:55:52.631703 13871 master_runner.cc:305] Clearing existing system tablet
I20260811 11:55:52.631974 13871 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:52.632031 13871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:52.633320 13871 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:55:52.634271 13982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:52.634507 13871 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:52.634768 13871 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:55:52.634863 13871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-1/wal
uuid: "79d3c932590e456c86fee8a8faecbec3"
format_stamp: "Formatted at 2026-08-11 11:55:52 on dist-test-slave-zkpd"
server_key: "ad96a12fa0c6bd049a8bb15dbcef2e92"
server_key_iv: "da00dde73a81b940e8cedf6c9949e8a4"
server_key_version: "encryptionkey@0"
I20260811 11:55:52.637044 13871 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:52.640782 13871 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 11:55:52.640858 13871 log.cc:1196] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 11:55:52.641141 13871 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3: Deleting consensus metadata
I20260811 11:55:52.641314 13871 master_runner.cc:315] Copying system tablet from 127.6.143.190:46619
I20260811 11:55:52.642544 13871 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3: tablet copy: Beginning tablet copy session from remote peer at address 127.6.143.190:46619
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.143.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.188.keytab.
Entry for principal oryx/127.6.143.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.188.keytab.
I20260811 11:55:52.646679 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.188:43931
--webserver_interface=127.6.143.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.188.keytab
--principal=oryx/127.6.143.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:45207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:46619,127.6.143.189:43751,127.6.143.188:43931
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:55:52.652019 13414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.643186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51160 (local address 127.6.143.190:46619)
0811 11:55:52.643332 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:52.643337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:52.643485 (+ 148us) server_negotiation.cc:407] Connection header received
0811 11:55:52.643621 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:52.643624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:52.643675 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:52.643759 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:52.644355 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.645185 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.646500 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.646865 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.646972 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.648445 (+ 1473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:52.648450 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:52.648452 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:52.648480 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:52.650165 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.650653 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.650657 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.650659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.650740 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.651043 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.651046 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.651048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.651257 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:52.651326 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:52.651674 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 11:55:52.651790 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 11:55:52.652554 13989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.643053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.0.0.1:51160)
0811 11:55:52.643377 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:52.643395 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:52.643523 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:52.643801 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:52.643807 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:52.644059 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:52.644153 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.644159 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.645296 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.645299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.646359 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.646373 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.647124 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.647128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.647224 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.647819 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:52.647836 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:52.648281 (+ 445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:52.650291 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.650294 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.650296 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.650542 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.650845 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.650848 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.650850 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.650943 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.651399 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:52.651402 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:52.651494 (+ 92us) client_negotiation.cc:769] Sending connection context
0811 11:55:52.652230 (+ 736us) client_negotiation.cc:240] Negotiation successful
0811 11:55:52.652372 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":91,"threads_started":1}
I20260811 11:55:52.653153 13394 tablet_copy_service.cc:140] P 8fedd24da8fe4ac59ffba44f177b1009: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 79d3c932590e456c86fee8a8faecbec3 ({username='oryx', principal='oryx/127.6.143.189@KRBTEST.COM'} at 127.0.0.1:51160)
I20260811 11:55:52.653378 13394 tablet_copy_service.cc:161] P 8fedd24da8fe4ac59ffba44f177b1009: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 79d3c932590e456c86fee8a8faecbec3 at {username='oryx', principal='oryx/127.6.143.189@KRBTEST.COM'} at 127.0.0.1:51160: session id = 79d3c932590e456c86fee8a8faecbec3-00000000000000000000000000000000
I20260811 11:55:52.655779 13394 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 11:55:52.657678 13871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 11:55:52.660128 13871 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3: tablet copy: Starting download of 0 data blocks...
I20260811 11:55:52.660315 13871 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3: tablet copy: Starting download of 1 WAL segments...
I20260811 11:55:52.662113 13871 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 11:55:52.663916 13394 tablet_copy_service.cc:342] P 8fedd24da8fe4ac59ffba44f177b1009: Request end of tablet copy session 79d3c932590e456c86fee8a8faecbec3-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.6.143.189@KRBTEST.COM'} at 127.0.0.1:51160
I20260811 11:55:52.664034 13394 tablet_copy_service.cc:433] P 8fedd24da8fe4ac59ffba44f177b1009: ending tablet copy session 79d3c932590e456c86fee8a8faecbec3-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 79d3c932590e456c86fee8a8faecbec3
I20260811 11:55:52.671214 13871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:52.672773 13991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:52.672870 13994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:52.673261 13871 server_base.cc:1034] running on GCE node
W20260811 11:55:52.676288 13992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:52.676510 13871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:52.676796 13871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:52.677945 13871 hybrid_clock.cc:648] HybridClock initialized: now 1786449352677921 us; error 38 us; skew 500 ppm
Aug 11 11:55:52 dist-test-slave-zkpd krb5kdc[13322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449352, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:52.679567 13871 init.cc:377] Logged in from keytab as oryx/127.6.143.189@KRBTEST.COM (short username oryx)
I20260811 11:55:52.680025 13871 webserver.cc:460] Webserver started at http://127.6.143.189:46235/ using document root <none> and password file <none>
I20260811 11:55:52.680213 13871 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:52.680256 13871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:52.681550 13871 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 11:55:52.682528 14003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:52.682770 13871 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:55:52.682890 13871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-1/wal
uuid: "79d3c932590e456c86fee8a8faecbec3"
format_stamp: "Formatted at 2026-08-11 11:55:52 on dist-test-slave-zkpd"
server_key: "ad96a12fa0c6bd049a8bb15dbcef2e92"
server_key_iv: "da00dde73a81b940e8cedf6c9949e8a4"
server_key_version: "encryptionkey@0"
I20260811 11:55:52.683027 13871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:52.690316 13871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:52.692358 13871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:52.697090 13871 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.189:43751
I20260811 11:55:52.697144 14060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.189:43751 every 8 connection(s)
I20260811 11:55:52.697831 13871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-1/data/info.pb
I20260811 11:55:52.700071 14061 sys_catalog.cc:263] Verifying existing consensus state
I20260811 11:55:52.701059 14061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3: Bootstrap starting.
I20260811 11:55:52.711006 14061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 11:55:52.711318 14061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3: Bootstrap complete.
I20260811 11:55:52.711871 14061 raft_consensus.cc:348] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: NON_VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: true } }
I20260811 11:55:52.712040 14061 raft_consensus.cc:729] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 79d3c932590e456c86fee8a8faecbec3, State: Initialized, Role: LEARNER
I20260811 11:55:52.712167 14061 consensus_queue.cc:260] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: NON_VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: true } }
I20260811 11:55:52.712546 14064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: NON_VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: true } } }
I20260811 11:55:52.712666 14061 sys_catalog.cc:564] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:55:52.712769 14064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [sys.catalog]: This master's current role is: LEARNER
I20260811 11:55:52.716809 14078 catalog_manager.cc:1259] Loaded cluster ID: e3111c441033471b8db261e6829e1e9e
I20260811 11:55:52.716868 14078 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3: loading cluster ID for follower catalog manager: success
I20260811 11:55:52.718860 14078 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3: acquiring CA information for follower catalog manager: success
I20260811 11:55:52.720274 14078 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 11:55:52.775116 13990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:52.775347 13990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:52.775418 13990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:52.779565 13990 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:55:52.779635 13990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:52.779655 13990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:52.779671 13990 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:55:52.779706 13990 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:55:52.785086 13990 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:45207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:46619,127.6.143.189:43751,127.6.143.188:43931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.188.keytab
--principal=oryx/127.6.143.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.188:43931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.143.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:52.786052 13990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:52.787041 13990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:52.795122 14083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:52.795257 13990 server_base.cc:1034] running on GCE node
W20260811 11:55:52.795158 14084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:52.795437 14086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:52.795990 13990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:52.797048 13990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:52.798233 13990 hybrid_clock.cc:648] HybridClock initialized: now 1786449352798219 us; error 24 us; skew 500 ppm
Aug 11 11:55:52 dist-test-slave-zkpd krb5kdc[13322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449352, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:52.800902 13990 init.cc:377] Logged in from keytab as oryx/127.6.143.188@KRBTEST.COM (short username oryx)
I20260811 11:55:52.801717 13990 webserver.cc:460] Webserver started at http://127.6.143.188:36017/ using document root <none> and password file <none>
I20260811 11:55:52.802300 13990 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:52.802417 13990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:52.802642 13990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:52.804718 13990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-2/data/instance:
uuid: "868a991373ad4c90a1e2aa593bc50755"
format_stamp: "Formatted at 2026-08-11 11:55:52 on dist-test-slave-zkpd"
server_key: "99de33e012263e1eae729d326bbb713c"
server_key_iv: "a78d550e15085c239a016b24e07ee066"
server_key_version: "encryptionkey@0"
I20260811 11:55:52.805267 13990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-2/wal/instance:
uuid: "868a991373ad4c90a1e2aa593bc50755"
format_stamp: "Formatted at 2026-08-11 11:55:52 on dist-test-slave-zkpd"
server_key: "99de33e012263e1eae729d326bbb713c"
server_key_iv: "a78d550e15085c239a016b24e07ee066"
server_key_version: "encryptionkey@0"
I20260811 11:55:52.808800 13990 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 11:55:52.811136 14094 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:52.812383 13990 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:55:52.812526 13990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-2/wal
uuid: "868a991373ad4c90a1e2aa593bc50755"
format_stamp: "Formatted at 2026-08-11 11:55:52 on dist-test-slave-zkpd"
server_key: "99de33e012263e1eae729d326bbb713c"
server_key_iv: "a78d550e15085c239a016b24e07ee066"
server_key_version: "encryptionkey@0"
I20260811 11:55:52.812630 13990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:52.832791 13990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:52.835657 13990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:52.835988 13990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:52.843148 13990 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.188:43931
I20260811 11:55:52.843456 14153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.188:43931 every 8 connection(s)
I20260811 11:55:52.844195 13990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-2/data/info.pb
I20260811 11:55:52.847126 14154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:52.851856 14154 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:55:52.854279 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 13990
Aug 11 11:55:52 dist-test-slave-zkpd krb5kdc[13322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449352, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.188@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM
Aug 11 11:55:52 dist-test-slave-zkpd krb5kdc[13322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449351, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.143.188@KRBTEST.COM
I20260811 11:55:52.866127 13414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.853818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51172 (local address 127.6.143.190:46619)
0811 11:55:52.853949 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:52.853953 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:52.854647 (+ 694us) server_negotiation.cc:407] Connection header received
0811 11:55:52.855792 (+ 1145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:52.855795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:52.855849 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:52.855986 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:52.857021 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.857831 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.858954 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.859209 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.859293 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.862103 (+ 2810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:52.862106 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:52.862108 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:52.862133 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:52.863611 (+ 1478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.864411 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.864415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.864417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.864516 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.864842 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.864846 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.864850 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.865060 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:52.865138 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:52.865828 (+ 690us) server_negotiation.cc:299] Negotiation successful
0811 11:55:52.865959 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 11:55:52.866912 14157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.854032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.0.0.1:51172)
0811 11:55:52.854487 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:52.854538 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:52.855584 (+ 1046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:52.856038 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:52.856048 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:52.856418 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:52.856867 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.856879 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.857978 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.857982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.858829 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.858840 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.859337 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.859339 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.859488 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.860134 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:52.860154 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:52.861971 (+ 1817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:52.863717 (+ 1746us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.863728 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.863738 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.864229 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.864617 (+ 388us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.864619 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.864620 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.864747 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.865168 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:52.865176 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:52.865488 (+ 312us) client_negotiation.cc:769] Sending connection context
0811 11:55:52.865731 (+ 243us) client_negotiation.cc:240] Negotiation successful
0811 11:55:52.866011 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":136,"threads_started":1}
I20260811 11:55:52.866914 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.855091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60806 (local address 127.6.143.188:43931)
0811 11:55:52.855432 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:52.855443 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:52.855484 (+ 41us) server_negotiation.cc:407] Connection header received
0811 11:55:52.855607 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:52.855621 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:52.855837 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:52.856778 (+ 941us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:52.856887 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.858121 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.859400 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.859794 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.859977 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.862798 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:52.862804 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:52.862817 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:52.862844 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:52.864826 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.865265 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.865270 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.865271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.865342 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.865750 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.865753 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.865755 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.866086 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:52.866508 (+ 422us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:52.866592 (+ 84us) server_negotiation.cc:299] Negotiation successful
0811 11:55:52.866758 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":113,"threads_started":1}
I20260811 11:55:52.868199 14154 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 11:55:52 dist-test-slave-zkpd krb5kdc[13322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449352, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.188@KRBTEST.COM for oryx/127.6.143.189@KRBTEST.COM
I20260811 11:55:52.876211 14159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.868842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38414 (local address 127.6.143.189:43751)
0811 11:55:52.869123 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:52.869127 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:52.869140 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:55:52.869183 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:52.869186 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:52.869235 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:52.869320 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:52.869671 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.870300 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.871007 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.871214 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.871288 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.873180 (+ 1892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:52.873183 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:52.873185 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:52.873207 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:52.874662 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.875131 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.875134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.875135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.875205 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.875450 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.875452 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.875453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.875608 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:52.875680 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:52.876007 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 11:55:52.876100 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":101,"threads_started":1}
I20260811 11:55:52.876212 14157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.868661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:43751 (local address 127.0.0.1:38414)
0811 11:55:52.868780 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:52.868792 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:52.868851 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:52.869362 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:52.869365 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:52.869487 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:52.869569 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.869573 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.870394 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.870396 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.870893 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.870899 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.871304 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.871306 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.871384 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.871827 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:52.871844 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:52.873053 (+ 1209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:52.874776 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.874780 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.874782 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.875028 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.875288 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.875291 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.875292 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.875377 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.875699 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:52.875702 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:52.875792 (+ 90us) client_negotiation.cc:769] Sending connection context
0811 11:55:52.875953 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 11:55:52.876072 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 11:55:52.877280 14154 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.188" port: 43931 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 11:55:52 dist-test-slave-zkpd krb5kdc[13322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449352, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.188@KRBTEST.COM for oryx/127.6.143.188@KRBTEST.COM
I20260811 11:55:52.884215 14157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.877721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:43931 (local address 127.0.0.1:60814)
0811 11:55:52.877803 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:52.877817 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:52.877894 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:52.878068 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:52.878070 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:52.878186 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:52.878246 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.878250 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.878980 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.878982 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.879446 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.879453 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.879983 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.879986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.880042 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.880486 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:52.880497 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:52.881552 (+ 1055us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:52.882942 (+ 1390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.882945 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.882946 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.883130 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.883344 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.883346 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.883347 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.883464 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.883796 (+ 332us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:52.883798 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:52.883876 (+ 78us) client_negotiation.cc:769] Sending connection context
0811 11:55:52.884033 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 11:55:52.884121 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260811 11:55:52.884341 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.877760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60814 (local address 127.6.143.188:43931)
0811 11:55:52.877866 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:52.877870 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:52.877886 (+ 16us) server_negotiation.cc:407] Connection header received
0811 11:55:52.877964 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:52.877966 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:52.877999 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:52.878055 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:52.878341 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.878876 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.879539 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.879834 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.879986 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.881676 (+ 1690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:52.881679 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:52.881681 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:52.881701 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:52.882846 (+ 1145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.883212 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.883214 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.883215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.883281 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.883566 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.883568 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.883569 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.883715 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:52.883790 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:52.884113 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 11:55:52.884221 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":12,"server-negotiator.queue_time_us":44}
I20260811 11:55:52.887341 14154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755: Bootstrap starting.
I20260811 11:55:52.889586 14154 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:52.890316 14154 log.cc:824] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:52.892153 14154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755: No bootstrap required, opened a new log
I20260811 11:55:52.895023 14154 raft_consensus.cc:348] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 43931 } }
I20260811 11:55:52.895248 14154 raft_consensus.cc:374] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:52.895332 14154 raft_consensus.cc:729] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 868a991373ad4c90a1e2aa593bc50755, State: Initialized, Role: FOLLOWER
I20260811 11:55:52.895766 14154 consensus_queue.cc:260] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 43931 } }
I20260811 11:55:52.896247 14161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 43931 } } }
I20260811 11:55:52.896355 14161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:55:52.896940 14154 sys_catalog.cc:564] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:55:52.898981 13577 heartbeater.cc:503] Master 127.6.143.190:46619 was elected leader, sending a full tablet report...
W20260811 11:55:52.901546 14173 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 11:55:52.901620 14173 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 11:55:52.904691 13990 master_runner.cc:428] Detected that this master 868a991373ad4c90a1e2aa593bc50755 is joining an existing cluster
I20260811 11:55:52.904747 13990 master_runner.cc:432] Initiating AddMaster RPC to add 127.6.143.188:43931
I20260811 11:55:52.916111 13414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.906781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51180 (local address 127.6.143.190:46619)
0811 11:55:52.906904 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:52.906908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:52.907216 (+ 308us) server_negotiation.cc:407] Connection header received
0811 11:55:52.907421 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:52.907426 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:52.907477 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:52.907528 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:52.908447 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.909256 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.910590 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.910928 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.910998 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.912609 (+ 1611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:52.912613 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:52.912615 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:52.912639 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:52.914364 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.914846 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.914850 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.914852 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.914944 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.915235 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.915238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.915239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.915424 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:52.915533 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:52.915844 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 11:55:52.915979 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 11:55:52.916123 14180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.906708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.0.0.1:51180)
0811 11:55:52.907122 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:52.907141 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:52.907290 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:52.907680 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:52.907683 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:52.908059 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:52.908306 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.908314 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.909384 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.909387 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.910441 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.910453 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.911149 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.911153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.911291 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.912004 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:52.912017 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:52.912480 (+ 463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:52.914483 (+ 2003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.914488 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.914490 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.914732 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.915040 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.915043 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.915045 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.915134 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.915524 (+ 390us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:52.915526 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:52.915645 (+ 119us) client_negotiation.cc:769] Sending connection context
0811 11:55:52.915773 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 11:55:52.915936 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"mutex_wait_us":182,"spinlock_wait_cycles":13440,"thread_start_us":235,"threads_started":1}
I20260811 11:55:52.920696 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 13438
Aug 11 11:55:52 dist-test-slave-zkpd krb5kdc[13322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449351, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.190@KRBTEST.COM for oryx/127.6.143.188@KRBTEST.COM
I20260811 11:55:52.928714 13969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.920111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:43931 (local address 127.0.0.1:60822)
0811 11:55:52.920225 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:52.920240 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:52.920313 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:52.920717 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:52.920721 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:52.920907 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:52.921017 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.921026 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.921790 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.921794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.922328 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.922336 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.922799 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.922803 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.922907 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.923347 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:52.923361 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:52.924890 (+ 1529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:52.927085 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.927092 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.927097 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.927327 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.927621 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.927626 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.927630 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.927713 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.928148 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:52.928153 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:52.928253 (+ 100us) client_negotiation.cc:769] Sending connection context
0811 11:55:52.928329 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 11:55:52.928487 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 11:55:52.929333 14183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.920146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60822 (local address 127.6.143.188:43931)
0811 11:55:52.920417 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:52.920424 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:52.920455 (+ 31us) server_negotiation.cc:407] Connection header received
0811 11:55:52.920522 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:52.920524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:52.920581 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:52.920703 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:52.921141 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.921678 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.922452 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.922685 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.922794 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.925291 (+ 2497us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:52.925297 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:52.925302 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:52.925331 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:52.926970 (+ 1639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.927447 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.927453 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.927458 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.927531 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.927809 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.927814 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.927819 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.928043 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:52.928905 (+ 862us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:52.928985 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 11:55:52.929103 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":144,"threads_started":1}
I20260811 11:55:52.933697 13371 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.6.143.188:43931: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "868a991373ad4c90a1e2aa593bc50755"
member_type: NON_VOTER
last_known_addr {
host: "127.6.143.188"
port: 43931
}
attrs {
promote: true
}
}
dest_uuid: "8fedd24da8fe4ac59ffba44f177b1009"
cas_config_opid_index: 5
I20260811 11:55:52.934320 13371 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: NON_VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: true } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: NON_VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: true } }
I20260811 11:55:52.934706 14182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.907459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:43931 (local address 127.0.0.1:60820)
0811 11:55:52.907967 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:52.907982 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:52.908119 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:52.908429 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:52.908431 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:52.908637 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:52.908737 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.908744 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.909612 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.909616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.910518 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.910527 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.911149 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.911153 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.911296 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.929969 (+ 18673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:52.929980 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:52.930258 (+ 278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:52.932937 (+ 2679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.932943 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.932948 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.933202 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.933488 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.933493 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.933498 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.933592 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.934050 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:52.934055 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:52.934150 (+ 95us) client_negotiation.cc:769] Sending connection context
0811 11:55:52.934246 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 11:55:52.934410 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"mutex_wait_us":89,"spinlock_wait_cycles":9216,"thread_start_us":77,"threads_started":1}
I20260811 11:55:52.935227 14158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.907292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60820 (local address 127.6.143.188:43931)
0811 11:55:52.907387 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:52.907391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:52.908055 (+ 664us) server_negotiation.cc:407] Connection header received
0811 11:55:52.908210 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:52.908214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:52.908259 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:52.908306 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:52.908846 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.909505 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.910618 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.910929 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.910998 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.930393 (+ 19395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:52.930407 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:52.930412 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:52.930441 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:52.932817 (+ 2376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.933301 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.933307 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.933311 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.933383 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.933696 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.933702 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.933706 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.933920 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:52.934015 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:52.934906 (+ 891us) server_negotiation.cc:299] Negotiation successful
0811 11:55:52.935024 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 11:55:52.935690 13976 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 868a991373ad4c90a1e2aa593bc50755 (127.6.143.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: NON_VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: true } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: NON_VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: true } } }
I20260811 11:55:52.937240 13976 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.6.143.188:43931
I20260811 11:55:52.937397 13976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8fedd24da8fe4ac59ffba44f177b1009" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: NON_VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: true } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: NON_VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: true } } }
I20260811 11:55:52.937489 13976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:52.938656 13990 master.cc:537] Master@127.6.143.188:43931 shutting down...
I20260811 11:55:52.940250 14181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.906777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:43751 (local address 127.0.0.1:38420)
0811 11:55:52.907234 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:52.907248 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:52.907366 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:52.907691 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:52.907693 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:52.908101 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:52.908304 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.908315 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.909397 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.909401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.910557 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.910569 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.911279 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.911283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.911392 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.930851 (+ 19459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:52.930871 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:52.931384 (+ 513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:52.937947 (+ 6563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.937951 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.937953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.938212 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.938546 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.938552 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.938557 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.938648 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.939333 (+ 685us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:52.939339 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:52.939457 (+ 118us) client_negotiation.cc:769] Sending connection context
0811 11:55:52.939575 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 11:55:52.939741 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"mutex_wait_us":112,"thread_start_us":112,"threads_started":1}
I20260811 11:55:52.940806 14159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.906762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38420 (local address 127.6.143.189:43751)
0811 11:55:52.906881 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:52.906886 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:52.907310 (+ 424us) server_negotiation.cc:407] Connection header received
0811 11:55:52.907468 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:52.907471 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:52.907518 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:52.907564 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:52.908453 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.909257 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.910699 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.911062 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.911161 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.931570 (+ 20409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:52.931577 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:52.931583 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:52.931615 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:52.937818 (+ 6203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.938332 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.938338 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.938342 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.938442 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.938749 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.938755 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.938760 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.938998 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:52.939104 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:52.940476 (+ 1372us) server_negotiation.cc:299] Negotiation successful
0811 11:55:52.940623 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
W20260811 11:55:52.941277 13355 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 -> Peer 868a991373ad4c90a1e2aa593bc50755 (127.6.143.188:43931): Couldn't send request to peer 868a991373ad4c90a1e2aa593bc50755. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 11:55:52.941956 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:39527
--local_ip_for_outbound_sockets=127.6.143.129
--tserver_master_addrs=127.6.143.190:46619
--webserver_port=35417
--webserver_interface=127.6.143.129
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.129.keytab
--principal=oryx/127.6.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:45207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.143.190:46619,127.6.143.189:43751
--tserver_master_addrs=127.6.143.190:46619,127.6.143.189:43751,127.6.143.188:43931 with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:55:52.946280 13969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.937781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:43751 (local address 127.0.0.1:38426)
0811 11:55:52.937935 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:52.937948 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:52.938016 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:52.941253 (+ 3237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:52.941256 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:52.941274 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:52.941375 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.941385 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.942443 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.942449 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.943801 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.943812 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.945834 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.945837 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.945925 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.945967 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 11:55:52.946045 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 11:55:52.946135 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 11:55:52.946558 13990 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260811 11:55:52.946619 14159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.937879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38426 (local address 127.6.143.189:43751)
0811 11:55:52.940964 (+ 3085us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:52.940970 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:52.940985 (+ 15us) server_negotiation.cc:407] Connection header received
0811 11:55:52.941030 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:52.941036 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:52.941096 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:52.941502 (+ 406us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:52.941610 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.942290 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.945126 (+ 2836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.945705 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.945820 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.945973 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:52.946454 (+ 481us) server_negotiation.cc:299] Negotiation successful
0811 11:55:52.946531 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3024,"thread_start_us":87,"threads_started":1}
I20260811 11:55:52.947012 13990 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:52.947059 14032 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [term 1 LEARNER]: Refusing update from remote peer 8fedd24da8fe4ac59ffba44f177b1009: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 11:55:52.947213 13990 tablet_replica.cc:330] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755: stopping tablet replica
I20260811 11:55:52.947532 14184 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: NON_VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 11:55:52.949461 14032 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 868a991373ad4c90a1e2aa593bc50755 (127.6.143.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: NON_VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: true } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: NON_VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: true } } }
I20260811 11:55:52.951433 14188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8fedd24da8fe4ac59ffba44f177b1009" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: NON_VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: true } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: NON_VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: true } } }
I20260811 11:55:52.951565 14188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [sys.catalog]: This master's current role is: LEARNER
I20260811 11:55:52.951874 14064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fedd24da8fe4ac59ffba44f177b1009. Latest consensus state: current_term: 1 leader_uuid: "8fedd24da8fe4ac59ffba44f177b1009" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: NON_VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: true } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: NON_VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: true } } }
I20260811 11:55:52.952075 14064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [sys.catalog]: This master's current role is: LEARNER
I20260811 11:55:52.952611 13990 master.cc:552] Master@127.6.143.188:43931 shutdown complete.
I20260811 11:55:52.952673 13990 master_runner.cc:305] Clearing existing system tablet
I20260811 11:55:52.952916 13990 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:52.952971 13990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:52.953239 14184 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009: attempting to promote NON_VOTER 79d3c932590e456c86fee8a8faecbec3 to VOTER
I20260811 11:55:52.953652 14184 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: false } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: NON_VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: true } }
I20260811 11:55:52.954661 13990 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 11:55:52.954807 13969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.954568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:43931 (local address 127.0.0.1:60828)
0811 11:55:52.954662 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:52.954738 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:43931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260811 11:55:52.954793 14032 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [term 1 LEARNER]: Refusing update from remote peer 8fedd24da8fe4ac59ffba44f177b1009: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 11:55:52.955363 13976 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 11:55:52.955444 13969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.955250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:43931 (local address 127.0.0.1:60844)
0811 11:55:52.955346 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:52.955388 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:43931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 11:55:52.955638 13355 consensus_peers.cc:487] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 -> Peer 868a991373ad4c90a1e2aa593bc50755 (127.6.143.188:43931): Couldn't send request to peer 868a991373ad4c90a1e2aa593bc50755. Status: Network error: Client connection negotiation failed: client connection to 127.6.143.188:43931: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 11:55:52.957140 14190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:52.957306 13976 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 79d3c932590e456c86fee8a8faecbec3 (127.6.143.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: false } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: NON_VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: true } } }
I20260811 11:55:52.958673 13976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8fedd24da8fe4ac59ffba44f177b1009" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: false } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: NON_VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: true } } }
I20260811 11:55:52.958774 13976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:52.958890 13976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8fedd24da8fe4ac59ffba44f177b1009" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: false } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: NON_VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: true } } }
I20260811 11:55:52.958968 13976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:52.959348 13990 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:52.959473 13990 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.000s
I20260811 11:55:52.959506 13990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-2/wal
uuid: "868a991373ad4c90a1e2aa593bc50755"
format_stamp: "Formatted at 2026-08-11 11:55:52 on dist-test-slave-zkpd"
server_key: "99de33e012263e1eae729d326bbb713c"
server_key_iv: "a78d550e15085c239a016b24e07ee066"
server_key_version: "encryptionkey@0"
I20260811 11:55:52.959513 14032 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 79d3c932590e456c86fee8a8faecbec3 (127.6.143.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: false } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: NON_VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: true } } }
I20260811 11:55:52.961541 14064 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8fedd24da8fe4ac59ffba44f177b1009" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: false } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: NON_VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: true } } }
I20260811 11:55:52.961654 14064 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:55:52.961836 13990 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:52.964322 13990 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 11:55:52.964413 13990 log.cc:1196] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 11:55:52.964733 13990 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755: Deleting consensus metadata
I20260811 11:55:52.964905 13990 master_runner.cc:315] Copying system tablet from 127.6.143.190:46619
I20260811 11:55:52.966053 13990 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755: tablet copy: Beginning tablet copy session from remote peer at address 127.6.143.190:46619
I20260811 11:55:52.984452 14196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.966534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.0.0.1:51188)
0811 11:55:52.976100 (+ 9566us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:52.976116 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:52.976236 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:52.976518 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:52.976521 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:52.976739 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:52.976871 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.976879 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.977927 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.977930 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.978815 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:52.978828 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.979420 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.979424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:52.979519 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.980177 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:52.980213 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:52.980641 (+ 428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:52.982615 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.982620 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.982622 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.982862 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.983221 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:52.983224 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:52.983226 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:52.983321 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:52.983790 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:52.983793 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:52.983942 (+ 149us) client_negotiation.cc:769] Sending connection context
0811 11:55:52.984115 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 11:55:52.984251 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9478,"thread_start_us":84,"threads_started":1}
I20260811 11:55:52.984469 13414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:52.966614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51188 (local address 127.6.143.190:46619)
0811 11:55:52.966719 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:52.966724 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:52.976211 (+ 9487us) server_negotiation.cc:407] Connection header received
0811 11:55:52.976334 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:52.976337 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:52.976391 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:52.976483 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:52.976993 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.977803 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.978951 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:52.979293 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:52.979395 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:52.980785 (+ 1390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:52.980789 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:52.980792 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:52.980818 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:52.982494 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.983026 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.983029 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.983031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.983123 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:52.983418 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:52.983421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:52.983423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:52.983649 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:52.983743 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:52.984199 (+ 456us) server_negotiation.cc:299] Negotiation successful
0811 11:55:52.984315 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 11:55:52.985229 13394 tablet_copy_service.cc:140] P 8fedd24da8fe4ac59ffba44f177b1009: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 868a991373ad4c90a1e2aa593bc50755 ({username='oryx', principal='oryx/127.6.143.188@KRBTEST.COM'} at 127.0.0.1:51188)
I20260811 11:55:52.985323 13394 tablet_copy_service.cc:161] P 8fedd24da8fe4ac59ffba44f177b1009: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 868a991373ad4c90a1e2aa593bc50755 at {username='oryx', principal='oryx/127.6.143.188@KRBTEST.COM'} at 127.0.0.1:51188: session id = 868a991373ad4c90a1e2aa593bc50755-00000000000000000000000000000000
I20260811 11:55:52.986383 13394 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 11:55:52.987761 13990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 11:55:52.990195 13990 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755: tablet copy: Starting download of 0 data blocks...
I20260811 11:55:52.990383 13990 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755: tablet copy: Starting download of 1 WAL segments...
I20260811 11:55:52.991770 13990 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 11:55:52.993433 13394 tablet_copy_service.cc:342] P 8fedd24da8fe4ac59ffba44f177b1009: Request end of tablet copy session 868a991373ad4c90a1e2aa593bc50755-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.6.143.188@KRBTEST.COM'} at 127.0.0.1:51188
I20260811 11:55:52.993520 13394 tablet_copy_service.cc:433] P 8fedd24da8fe4ac59ffba44f177b1009: ending tablet copy session 868a991373ad4c90a1e2aa593bc50755-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 868a991373ad4c90a1e2aa593bc50755
I20260811 11:55:53.000721 13990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:53.002347 14197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:53.004721 14198 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:53.005008 14200 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:53.008693 13990 server_base.cc:1034] running on GCE node
I20260811 11:55:53.008844 13990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:53.009146 13990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:53.010298 13990 hybrid_clock.cc:648] HybridClock initialized: now 1786449353010272 us; error 38 us; skew 500 ppm
Aug 11 11:55:53 dist-test-slave-zkpd krb5kdc[13322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449353, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:53.011983 13990 init.cc:377] Logged in from keytab as oryx/127.6.143.188@KRBTEST.COM (short username oryx)
I20260811 11:55:53.012383 13990 webserver.cc:460] Webserver started at http://127.6.143.188:36777/ using document root <none> and password file <none>
I20260811 11:55:53.012568 13990 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:53.012606 13990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:53.014046 13990 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:55:53.015066 14207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:53.015345 13990 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 11:55:53.015424 13990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-2/wal
uuid: "868a991373ad4c90a1e2aa593bc50755"
format_stamp: "Formatted at 2026-08-11 11:55:52 on dist-test-slave-zkpd"
server_key: "99de33e012263e1eae729d326bbb713c"
server_key_iv: "a78d550e15085c239a016b24e07ee066"
server_key_version: "encryptionkey@0"
I20260811 11:55:53.015501 13990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:53.049448 13990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:53.051651 13990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:53.056969 13990 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.188:43931
I20260811 11:55:53.057076 14264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.188:43931 every 8 connection(s)
I20260811 11:55:53.057590 13990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/master-2/data/info.pb
I20260811 11:55:53.059479 14265 sys_catalog.cc:263] Verifying existing consensus state
I20260811 11:55:53.060776 14265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755: Bootstrap starting.
I20260811 11:55:53.072504 14265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 11:55:53.072810 14265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755: Bootstrap complete.
I20260811 11:55:53.073371 14265 raft_consensus.cc:348] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: false } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: NON_VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: true } }
I20260811 11:55:53.073535 14265 raft_consensus.cc:729] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 868a991373ad4c90a1e2aa593bc50755, State: Initialized, Role: LEARNER
I20260811 11:55:53.073658 14265 consensus_queue.cc:260] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: false } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: NON_VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: true } }
I20260811 11:55:53.074012 14268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: false } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: NON_VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: true } } }
I20260811 11:55:53.074162 14265 sys_catalog.cc:564] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:55:53.074246 14268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [sys.catalog]: This master's current role is: LEARNER
I20260811 11:55:53.078059 14279 catalog_manager.cc:1259] Loaded cluster ID: e3111c441033471b8db261e6829e1e9e
I20260811 11:55:53.078114 14279 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755: loading cluster ID for follower catalog manager: success
I20260811 11:55:53.080435 14279 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755: acquiring CA information for follower catalog manager: success
I20260811 11:55:53.081790 14279 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 11:55:53.089147 14186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:53.089392 14186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:53.089469 14186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:53.093976 14186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:53.094043 14186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:53.094141 14186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:55:53.099704 14186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:45207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.129.keytab
--principal=oryx/127.6.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:39527
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=35417
--tserver_master_addrs=127.6.143.190:46619,127.6.143.189:43751,127.6.143.188:43931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:53.100677 14186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:53.101641 14186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:53.111147 14288 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:53.111207 14186 server_base.cc:1034] running on GCE node
W20260811 11:55:53.111160 14285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:53.111155 14291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:53.111989 14186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:53.112769 14186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:53.113202 13721 heartbeater.cc:503] Master 127.6.143.190:46619 was elected leader, sending a full tablet report...
I20260811 11:55:53.113953 14186 hybrid_clock.cc:648] HybridClock initialized: now 1786449353113909 us; error 59 us; skew 500 ppm
Aug 11 11:55:53 dist-test-slave-zkpd krb5kdc[13322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449353, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:53.116705 14186 init.cc:377] Logged in from keytab as oryx/127.6.143.129@KRBTEST.COM (short username oryx)
I20260811 11:55:53.117496 14186 webserver.cc:460] Webserver started at http://127.6.143.129:35417/ using document root <none> and password file <none>
I20260811 11:55:53.118032 14186 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:53.118129 14186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:53.121750 14186 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:55:53.123643 14299 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:53.124879 14186 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:53.125017 14186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "b063e255083e4cdda907e958091e2606"
format_stamp: "Formatted at 2026-08-11 11:55:51 on dist-test-slave-zkpd"
server_key: "67d15f2befefda95e9b5a829cfbeb17e"
server_key_iv: "81d26e395458042eb718feb7c55a47c8"
server_key_version: "encryptionkey@0"
I20260811 11:55:53.125533 14186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:53.136241 14186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:53.138849 14186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:53.138988 14186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:53.139473 14186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:55:53.140314 14186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:53.140360 14186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:53.140439 14186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:53.140481 14186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:53.149503 14186 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:39527
I20260811 11:55:53.149528 14422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:39527 every 8 connection(s)
I20260811 11:55:53.150451 14186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:55:53.150797 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 14186
I20260811 11:55:53.151016 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 13582
Aug 11 11:55:53 dist-test-slave-zkpd krb5kdc[13322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449353, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM
I20260811 11:55:53.165907 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:42017
--local_ip_for_outbound_sockets=127.6.143.130
--tserver_master_addrs=127.6.143.190:46619
--webserver_port=39337
--webserver_interface=127.6.143.130
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.130.keytab
--principal=oryx/127.6.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:45207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.143.190:46619,127.6.143.189:43751
--tserver_master_addrs=127.6.143.190:46619,127.6.143.189:43751,127.6.143.188:43931 with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 11:55:53 dist-test-slave-zkpd krb5kdc[13322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449353, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.189@KRBTEST.COM
Aug 11 11:55:53 dist-test-slave-zkpd krb5kdc[13322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449353, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.188@KRBTEST.COM
I20260811 11:55:53.168567 13414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.154839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:32983 (local address 127.6.143.190:46619)
0811 11:55:53.155013 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:53.155020 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:53.156603 (+ 1583us) server_negotiation.cc:407] Connection header received
0811 11:55:53.156930 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:53.156936 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:53.157002 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:53.157063 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:53.158898 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.159838 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.161163 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.161399 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.161515 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.164050 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:53.164053 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:53.164056 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:53.164087 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:53.165931 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:53.166582 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:53.166588 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:53.166593 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:53.166686 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:53.167117 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:53.167124 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:53.167128 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:53.167400 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:53.167482 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:53.168256 (+ 774us) server_negotiation.cc:299] Negotiation successful
0811 11:55:53.168424 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 11:55:53.170375 14432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.154294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.6.143.129:32983)
0811 11:55:53.156506 (+ 2212us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:53.156524 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:53.156697 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:53.157478 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:53.157490 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:53.158018 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:53.158722 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.158745 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.160069 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.160075 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.161023 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.161034 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.161555 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.161560 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.161737 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.162270 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:53.162285 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:53.163827 (+ 1542us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:53.166047 (+ 2220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:53.166054 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:53.166075 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:53.166451 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:53.166807 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:53.166812 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:53.166817 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:53.166999 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:53.167576 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:53.167584 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:53.167939 (+ 355us) client_negotiation.cc:769] Sending connection context
0811 11:55:53.168818 (+ 879us) client_negotiation.cc:240] Negotiation successful
0811 11:55:53.169230 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2108,"mutex_wait_us":16,"thread_start_us":228,"threads_started":1}
I20260811 11:55:53.170858 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.155393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:42407 (local address 127.6.143.188:43931)
0811 11:55:53.155671 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:53.155677 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:53.155696 (+ 19us) server_negotiation.cc:407] Connection header received
0811 11:55:53.157711 (+ 2015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:53.157717 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:53.157782 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:53.157857 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:53.159119 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.160304 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.163922 (+ 3618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.164395 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.164486 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.166972 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:53.166977 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:53.166980 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:53.167007 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:53.168796 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:53.169425 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:53.169429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:53.169430 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:53.169518 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:53.169855 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:53.169858 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:53.169860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:53.170090 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:53.170166 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:53.170579 (+ 413us) server_negotiation.cc:299] Negotiation successful
0811 11:55:53.170700 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":86,"threads_started":1}
I20260811 11:55:53.171136 14429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.152826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:43751 (local address 127.6.143.129:52877)
0811 11:55:53.156292 (+ 3466us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:53.156394 (+ 102us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:53.156885 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:53.157478 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:53.157490 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:53.158622 (+ 1132us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:53.158777 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.158789 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.162561 (+ 3772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.162567 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.163535 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.163551 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.164299 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.164304 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.164436 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.164834 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:53.164850 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:53.167020 (+ 2170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:53.169007 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:53.169010 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:53.169012 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:53.169322 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:53.169627 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:53.169630 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:53.169632 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:53.169745 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:53.170225 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:53.170228 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:53.170357 (+ 129us) client_negotiation.cc:769] Sending connection context
0811 11:55:53.170494 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 11:55:53.170627 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3366,"mutex_wait_us":40,"spinlock_wait_cycles":54016,"thread_start_us":484,"threads_started":1}
I20260811 11:55:53.171423 14428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.152826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:43931 (local address 127.6.143.129:42407)
0811 11:55:53.155426 (+ 2600us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:53.155487 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:53.156693 (+ 1206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:53.158031 (+ 1338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:53.158036 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:53.158261 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:53.158979 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.158992 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.160486 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.160492 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.163763 (+ 3271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.163776 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.164830 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.164837 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.164964 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.165443 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:53.165461 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:53.166786 (+ 1325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:53.168942 (+ 2156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:53.168948 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:53.168951 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:53.169303 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:53.169630 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:53.169632 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:53.169634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:53.169751 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:53.170225 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:53.170228 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:53.170357 (+ 129us) client_negotiation.cc:769] Sending connection context
0811 11:55:53.171082 (+ 725us) client_negotiation.cc:240] Negotiation successful
0811 11:55:53.171228 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2374,"mutex_wait_us":461,"spinlock_wait_cycles":8320,"thread_start_us":507,"threads_started":1}
I20260811 11:55:53.171664 14423 heartbeater.cc:347] Connected to a master server at 127.6.143.190:46619
I20260811 11:55:53.171733 14159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.155233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:52877 (local address 127.6.143.189:43751)
0811 11:55:53.155396 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:53.155410 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:53.156477 (+ 1067us) server_negotiation.cc:407] Connection header received
0811 11:55:53.157006 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:53.157012 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:53.157082 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:53.157146 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:53.161596 (+ 4450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.162410 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.163676 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.164055 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.164159 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.167172 (+ 3013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:53.167176 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:53.167179 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:53.167213 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:53.168898 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:53.169436 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:53.169440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:53.169442 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:53.169518 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:53.169855 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:53.169858 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:53.169860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:53.170090 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:53.170180 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:53.171440 (+ 1260us) server_negotiation.cc:299] Negotiation successful
0811 11:55:53.171560 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 11:55:53.171939 14423 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:53.172395 14424 heartbeater.cc:347] Connected to a master server at 127.6.143.188:43931
I20260811 11:55:53.172509 14424 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:53.172540 14425 heartbeater.cc:347] Connected to a master server at 127.6.143.189:43751
I20260811 11:55:53.172513 14423 heartbeater.cc:511] Master 127.6.143.190:46619 requested a full tablet report, sending...
I20260811 11:55:53.172644 14425 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:53.172703 14424 heartbeater.cc:511] Master 127.6.143.188:43931 requested a full tablet report, sending...
I20260811 11:55:53.172817 14425 heartbeater.cc:511] Master 127.6.143.189:43751 requested a full tablet report, sending...
I20260811 11:55:53.174000 13371 ts_manager.cc:194] Re-registered known tserver with Master: b063e255083e4cdda907e958091e2606 (127.6.143.129:39527)
I20260811 11:55:53.174165 14022 ts_manager.cc:194] Registered new tserver with Master: b063e255083e4cdda907e958091e2606 (127.6.143.129:39527)
I20260811 11:55:53.174460 14225 ts_manager.cc:194] Registered new tserver with Master: b063e255083e4cdda907e958091e2606 (127.6.143.129:39527)
I20260811 11:55:53.174721 13371 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.143.129@KRBTEST.COM'} at 127.6.143.129:32983
W20260811 11:55:53.286875 14434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:53.287102 14434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:53.287143 14434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:53.291075 14434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:53.291131 14434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:53.291185 14434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:55:53.296253 14434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:45207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.130.keytab
--principal=oryx/127.6.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:42017
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=39337
--tserver_master_addrs=127.6.143.190:46619,127.6.143.189:43751,127.6.143.188:43931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:53.297272 14434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:53.298305 14434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:53.307971 14443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:53.307988 14445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:53.308066 14434 server_base.cc:1034] running on GCE node
W20260811 11:55:53.308091 14442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:53.308597 14434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:53.309217 14434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:53.310431 14434 hybrid_clock.cc:648] HybridClock initialized: now 1786449353310416 us; error 26 us; skew 500 ppm
Aug 11 11:55:53 dist-test-slave-zkpd krb5kdc[13322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449353, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:53.313657 14434 init.cc:377] Logged in from keytab as oryx/127.6.143.130@KRBTEST.COM (short username oryx)
I20260811 11:55:53.314698 14434 webserver.cc:460] Webserver started at http://127.6.143.130:39337/ using document root <none> and password file <none>
I20260811 11:55:53.315313 14434 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:53.315393 14434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:53.319358 14434 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260811 11:55:53.321475 14452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:53.322660 14434 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 11:55:53.322993 14434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "0e664a94aff147c5aea4d0c580f610fe"
format_stamp: "Formatted at 2026-08-11 11:55:52 on dist-test-slave-zkpd"
server_key: "d0ae82e29b01275185d3374cff2f3c5d"
server_key_iv: "83adbed30053a3e7bb8c8dd4427a2854"
server_key_version: "encryptionkey@0"
I20260811 11:55:53.323539 14434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:53.331962 13865 heartbeater.cc:503] Master 127.6.143.190:46619 was elected leader, sending a full tablet report...
I20260811 11:55:53.337234 13969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.333894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:43931 (local address 127.0.0.1:60854)
0811 11:55:53.334019 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:53.334039 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:53.334145 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:53.334423 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:53.334426 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:53.334443 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:53.334550 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.334555 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.335305 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.335310 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.336125 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.336135 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.336843 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.336847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.336942 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.336966 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 11:55:53.337034 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 11:55:53.337108 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 11:55:53.337234 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.334016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60854 (local address 127.6.143.188:43931)
0811 11:55:53.334159 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:53.334164 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:53.334180 (+ 16us) server_negotiation.cc:407] Connection header received
0811 11:55:53.334240 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:53.334243 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:53.334306 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:53.334412 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:53.334667 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.335190 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.336250 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.336717 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.336843 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.336998 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:53.337044 (+ 46us) server_negotiation.cc:299] Negotiation successful
0811 11:55:53.337111 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 11:55:53.338129 14237 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 11:55:53.338599 14268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fedd24da8fe4ac59ffba44f177b1009. Latest consensus state: current_term: 1 leader_uuid: "8fedd24da8fe4ac59ffba44f177b1009" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: false } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: NON_VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: true } } }
I20260811 11:55:53.338739 14268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [sys.catalog]: This master's current role is: LEARNER
I20260811 11:55:53.344038 14434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:53.346805 14434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:53.347016 14434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:53.347599 14434 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:55:53.348568 14434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:53.348618 14434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:53.348658 14434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:53.348676 14434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:53.357486 14434 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:42017
I20260811 11:55:53.357511 14575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:42017 every 8 connection(s)
I20260811 11:55:53.358516 14434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
I20260811 11:55:53.368142 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 14434
I20260811 11:55:53.368324 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 13727
Aug 11 11:55:53 dist-test-slave-zkpd krb5kdc[13322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449353, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.188@KRBTEST.COM
Aug 11 11:55:53 dist-test-slave-zkpd krb5kdc[13322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449353, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.189@KRBTEST.COM
I20260811 11:55:53.376019 14159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.360319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:52405 (local address 127.6.143.189:43751)
0811 11:55:53.360482 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:53.360488 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:53.361394 (+ 906us) server_negotiation.cc:407] Connection header received
0811 11:55:53.362569 (+ 1175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:53.362573 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:53.362626 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:53.362680 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:53.364222 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.364992 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.366430 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.366653 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.366769 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.371528 (+ 4759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:53.371531 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:53.371534 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:53.371566 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:53.373456 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:53.374111 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:53.374116 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:53.374120 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:53.374199 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:53.374608 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:53.374626 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:53.374630 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:53.374870 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:53.374942 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:53.375649 (+ 707us) server_negotiation.cc:299] Negotiation successful
0811 11:55:53.375780 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 11:55:53.376171 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.360308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:34061 (local address 127.6.143.188:43931)
0811 11:55:53.360456 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:53.360461 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:53.362085 (+ 1624us) server_negotiation.cc:407] Connection header received
0811 11:55:53.362568 (+ 483us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:53.362572 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:53.362626 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:53.362680 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:53.364208 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.364714 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.366438 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.366771 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.366842 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.371017 (+ 4175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:53.371020 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:53.371022 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:53.371045 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:53.373202 (+ 2157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:53.374171 (+ 969us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:53.374177 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:53.374182 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:53.374295 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:53.374718 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:53.374725 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:53.374730 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:53.375040 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:53.375130 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:53.375701 (+ 571us) server_negotiation.cc:299] Negotiation successful
0811 11:55:53.375876 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 11:55:53.378046 14582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.360755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:43751 (local address 127.6.143.130:52405)
0811 11:55:53.361227 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:53.361276 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:53.362336 (+ 1060us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:53.362871 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:53.362879 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:53.363340 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:53.364046 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.364056 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.365104 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.365107 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.366302 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.366311 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.367162 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.367165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.367271 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.368093 (+ 822us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:53.368104 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:53.371369 (+ 3265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:53.373621 (+ 2252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:53.373627 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:53.373631 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:53.374003 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:53.374302 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:53.374307 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:53.374311 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:53.374500 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:53.375038 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:53.375046 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:53.375394 (+ 348us) client_negotiation.cc:769] Sending connection context
0811 11:55:53.376585 (+ 1191us) client_negotiation.cc:240] Negotiation successful
0811 11:55:53.376887 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":227,"spinlock_wait_cycles":35328,"thread_start_us":148,"threads_started":1}
I20260811 11:55:53.378046 14584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.360755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:43931 (local address 127.6.143.130:34061)
0811 11:55:53.361973 (+ 1218us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:53.361987 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:53.362336 (+ 349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:53.362877 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:53.362880 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:53.363343 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:53.364046 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.364056 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.364860 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.364872 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.366303 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.366313 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.367039 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.367043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.367242 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.367832 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:53.367853 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:53.370821 (+ 2968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:53.373366 (+ 2545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:53.373372 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:53.373393 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:53.374021 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:53.374441 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:53.374447 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:53.374458 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:53.374581 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:53.375228 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:53.375234 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:53.375400 (+ 166us) client_negotiation.cc:769] Sending connection context
0811 11:55:53.376438 (+ 1038us) client_negotiation.cc:240] Negotiation successful
0811 11:55:53.377005 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1126,"mutex_wait_us":389,"spinlock_wait_cycles":23808,"thread_start_us":149,"threads_started":1}
Aug 11 11:55:53 dist-test-slave-zkpd krb5kdc[13322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449353, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM
I20260811 11:55:53.379933 14578 heartbeater.cc:347] Connected to a master server at 127.6.143.188:43931
I20260811 11:55:53.380187 14578 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:53.380771 14578 heartbeater.cc:511] Master 127.6.143.188:43931 requested a full tablet report, sending...
I20260811 11:55:53.382192 14225 ts_manager.cc:194] Registered new tserver with Master: 0e664a94aff147c5aea4d0c580f610fe (127.6.143.130:42017)
I20260811 11:55:53.383440 13414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.361799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:42925 (local address 127.6.143.190:46619)
0811 11:55:53.361938 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:53.361943 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:53.362138 (+ 195us) server_negotiation.cc:407] Connection header received
0811 11:55:53.362541 (+ 403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:53.362544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:53.362595 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:53.362681 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:53.364181 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.364986 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.367173 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.367538 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.367653 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.379543 (+ 11890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:53.379551 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:53.379557 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:53.379608 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:53.381605 (+ 1997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:53.382094 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:53.382101 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:53.382105 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:53.382225 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:53.382595 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:53.382598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:53.382600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:53.382781 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:53.382914 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:53.383189 (+ 275us) server_negotiation.cc:299] Negotiation successful
0811 11:55:53.383304 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 11:55:53.383759 14583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.361676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.6.143.130:42925)
0811 11:55:53.362034 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:53.362066 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:53.362347 (+ 281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:53.362870 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:53.362877 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:53.363323 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:53.364039 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.364055 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.366213 (+ 2158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.366216 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.367055 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.367062 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.367696 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.367702 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.367821 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.377852 (+ 10031us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:53.377880 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:53.379372 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:53.381723 (+ 2351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:53.381728 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:53.381730 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:53.381976 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:53.382358 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:53.382361 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:53.382363 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:53.382457 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:53.382903 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:53.382906 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:53.382996 (+ 90us) client_negotiation.cc:769] Sending connection context
0811 11:55:53.383131 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 11:55:53.383266 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":118,"thread_start_us":88,"threads_started":1}
I20260811 11:55:53.384604 14576 heartbeater.cc:347] Connected to a master server at 127.6.143.190:46619
I20260811 11:55:53.384681 14576 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:53.384814 14576 heartbeater.cc:511] Master 127.6.143.190:46619 requested a full tablet report, sending...
I20260811 11:55:53.385396 13371 ts_manager.cc:194] Re-registered known tserver with Master: 0e664a94aff147c5aea4d0c580f610fe (127.6.143.130:42017)
I20260811 11:55:53.386284 13371 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.143.130@KRBTEST.COM'} at 127.6.143.130:42925
I20260811 11:55:53.388624 14579 heartbeater.cc:347] Connected to a master server at 127.6.143.189:43751
I20260811 11:55:53.388715 14579 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:53.388895 14579 heartbeater.cc:511] Master 127.6.143.189:43751 requested a full tablet report, sending...
I20260811 11:55:53.389573 14022 ts_manager.cc:194] Registered new tserver with Master: 0e664a94aff147c5aea4d0c580f610fe (127.6.143.130:42017)
I20260811 11:55:53.390771 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:42641
--local_ip_for_outbound_sockets=127.6.143.131
--tserver_master_addrs=127.6.143.190:46619
--webserver_port=43327
--webserver_interface=127.6.143.131
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.131.keytab
--principal=oryx/127.6.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:45207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.143.190:46619,127.6.143.189:43751
--tserver_master_addrs=127.6.143.190:46619,127.6.143.189:43751,127.6.143.188:43931 with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:53.505791 14585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:53.506058 14585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:53.506156 14585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:53.510074 14585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:53.510151 14585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:53.510238 14585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:55:53.515796 14585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:45207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.131.keytab
--principal=oryx/127.6.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:42641
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=43327
--tserver_master_addrs=127.6.143.190:46619,127.6.143.189:43751,127.6.143.188:43931
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14585
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:53.516821 14585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:53.517769 14585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:53.527292 14593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:53.527374 14590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:53.527520 14585 server_base.cc:1034] running on GCE node
W20260811 11:55:53.527290 14591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:53.528057 14585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:53.528712 14585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:53.529915 14585 hybrid_clock.cc:648] HybridClock initialized: now 1786449353529891 us; error 28 us; skew 500 ppm
Aug 11 11:55:53 dist-test-slave-zkpd krb5kdc[13322](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449353, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:53.532585 14585 init.cc:377] Logged in from keytab as oryx/127.6.143.131@KRBTEST.COM (short username oryx)
I20260811 11:55:53.533346 14585 webserver.cc:460] Webserver started at http://127.6.143.131:43327/ using document root <none> and password file <none>
I20260811 11:55:53.533891 14585 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:53.533962 14585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:53.538007 14585 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 11:55:53.540066 14600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:53.541299 14585 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:53.541407 14585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "d8af8d74ff8b451e80081f46f94504a2"
format_stamp: "Formatted at 2026-08-11 11:55:52 on dist-test-slave-zkpd"
server_key: "e55eb35271deefff0b9fa4949150ccbe"
server_key_iv: "5aead28516a814495887373f58043f42"
server_key_version: "encryptionkey@0"
I20260811 11:55:53.541913 14585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:53.559755 14585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:53.562935 14585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:53.563130 14585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:53.563738 14585 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:55:53.564641 14585 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:53.564689 14585 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:53.564757 14585 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:53.564786 14585 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:53.573659 14585 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:42641
I20260811 11:55:53.573700 14723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:42641 every 8 connection(s)
I20260811 11:55:53.574831 14585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:55:53.577381 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 14585
Aug 11 11:55:53 dist-test-slave-zkpd krb5kdc[13322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449353, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM
Aug 11 11:55:53 dist-test-slave-zkpd krb5kdc[13322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449353, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.189@KRBTEST.COM
I20260811 11:55:53.594357 14159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.576748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:33937 (local address 127.6.143.189:43751)
0811 11:55:53.576869 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:53.576874 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:53.579775 (+ 2901us) server_negotiation.cc:407] Connection header received
0811 11:55:53.579826 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:53.579829 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:53.579881 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:53.579981 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:53.581871 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.582671 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.584681 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.585023 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.585097 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.590330 (+ 5233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:53.590335 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:53.590339 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:53.590368 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:53.592332 (+ 1964us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:53.592806 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:53.592810 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:53.592812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:53.592908 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:53.593217 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:53.593221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:53.593222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:53.593429 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:53.593499 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:53.594043 (+ 544us) server_negotiation.cc:299] Negotiation successful
0811 11:55:53.594168 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Aug 11 11:55:53 dist-test-slave-zkpd krb5kdc[13322](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449353, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.188@KRBTEST.COM
I20260811 11:55:53.594726 14738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.579638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38436 (local address 127.6.143.189:43751)
0811 11:55:53.580926 (+ 1288us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:53.580931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:53.581537 (+ 606us) server_negotiation.cc:407] Connection header received
0811 11:55:53.582109 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:53.582112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:53.582161 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:53.582223 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:53.584171 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.584916 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.587263 (+ 2347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.587634 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.587708 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.590415 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:53.590419 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:53.590422 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:53.590455 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:53.592431 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:53.593073 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:53.593076 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:53.593078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:53.593156 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:53.593432 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:53.593435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:53.593437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:53.593614 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:53.593734 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:53.594461 (+ 727us) server_negotiation.cc:299] Negotiation successful
0811 11:55:53.594575 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1199,"thread_start_us":70,"threads_started":1}
I20260811 11:55:53.595750 13414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.576715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:39689 (local address 127.6.143.190:46619)
0811 11:55:53.576883 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:53.576888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:53.578289 (+ 1401us) server_negotiation.cc:407] Connection header received
0811 11:55:53.579830 (+ 1541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:53.579833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:53.579881 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:53.579964 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:53.581862 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.582674 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.584681 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.585023 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.585105 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.588608 (+ 3503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:53.588612 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:53.588615 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:53.588642 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:53.591785 (+ 3143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:53.592464 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:53.592468 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:53.592470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:53.592552 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:53.592975 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:53.592981 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:53.592987 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:53.593216 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:53.593296 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:53.595229 (+ 1933us) server_negotiation.cc:299] Negotiation successful
0811 11:55:53.595342 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":68}
I20260811 11:55:53.596215 14728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.577138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:43751 (local address 127.6.143.131:33937)
0811 11:55:53.579487 (+ 2349us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:53.579502 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:53.579598 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:53.580266 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:53.580275 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:53.581143 (+ 868us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:53.581682 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.581703 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.582869 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.582872 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.584539 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.584551 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.585310 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.585314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.585666 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.588595 (+ 2929us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:53.588616 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:53.590165 (+ 1549us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:53.592454 (+ 2289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:53.592457 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:53.592459 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:53.592698 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:53.593024 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:53.593029 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:53.593034 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:53.593109 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:53.593570 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:53.593572 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:53.593712 (+ 140us) client_negotiation.cc:769] Sending connection context
0811 11:55:53.594506 (+ 794us) client_negotiation.cc:240] Negotiation successful
0811 11:55:53.594850 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2282,"mutex_wait_us":453,"spinlock_wait_cycles":10368,"thread_start_us":149,"threads_started":1}
I20260811 11:55:53.596737 14729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.577120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.6.143.131:39689)
0811 11:55:53.578115 (+ 995us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:53.578168 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:53.579169 (+ 1001us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:53.580266 (+ 1097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:53.580275 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:53.580857 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:53.581683 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.581703 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.582868 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.582871 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.584539 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.584551 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.585310 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.585314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.585666 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.586531 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:53.586550 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:53.588454 (+ 1904us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:53.591964 (+ 3510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:53.591972 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:53.591989 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:53.592341 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:53.592675 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:53.592678 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:53.592679 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:53.592856 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:53.593374 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:53.593380 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:53.593712 (+ 332us) client_negotiation.cc:769] Sending connection context
0811 11:55:53.596184 (+ 2472us) client_negotiation.cc:240] Negotiation successful
0811 11:55:53.596555 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":794,"mutex_wait_us":299,"spinlock_wait_cycles":8576,"thread_start_us":94,"threads_started":1}
I20260811 11:55:53.597515 14726 heartbeater.cc:347] Connected to a master server at 127.6.143.189:43751
I20260811 11:55:53.597750 14726 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:53.597981 14724 heartbeater.cc:347] Connected to a master server at 127.6.143.190:46619
I20260811 11:55:53.598078 14724 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:53.598312 14724 heartbeater.cc:511] Master 127.6.143.190:46619 requested a full tablet report, sending...
I20260811 11:55:53.598308 14726 heartbeater.cc:511] Master 127.6.143.189:43751 requested a full tablet report, sending...
I20260811 11:55:53.599445 13371 ts_manager.cc:194] Re-registered known tserver with Master: d8af8d74ff8b451e80081f46f94504a2 (127.6.143.131:42641)
I20260811 11:55:53.600152 13371 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.143.131@KRBTEST.COM'} at 127.6.143.131:39689
I20260811 11:55:53.600613 14022 ts_manager.cc:194] Registered new tserver with Master: d8af8d74ff8b451e80081f46f94504a2 (127.6.143.131:42641)
I20260811 11:55:53.601488 14735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.579410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51204 (local address 127.6.143.190:46619)
0811 11:55:53.581058 (+ 1648us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:53.581062 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:53.581303 (+ 241us) server_negotiation.cc:407] Connection header received
0811 11:55:53.581827 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:53.581832 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:53.581890 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:53.581968 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:53.585042 (+ 3074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.585854 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.589291 (+ 3437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.589653 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.589767 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.595219 (+ 5452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:53.595223 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:53.595225 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:53.595252 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:53.596962 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:53.598289 (+ 1327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:53.598292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:53.598294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:53.598381 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:53.598661 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:53.598664 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:53.598666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:53.598843 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:53.600878 (+ 2035us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:53.601221 (+ 343us) server_negotiation.cc:299] Negotiation successful
0811 11:55:53.601342 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1584,"thread_start_us":78,"threads_started":1}
I20260811 11:55:53.603396 14730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.578020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:43931 (local address 127.6.143.131:43821)
0811 11:55:53.580634 (+ 2614us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:53.580650 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:53.580755 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:53.581044 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:53.581047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:53.581245 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:53.581797 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.581808 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.582874 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.582877 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.586823 (+ 3946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.586835 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.587594 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.587598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.587711 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.588168 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:53.588182 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:53.599467 (+ 11285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:53.601717 (+ 2250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:53.601721 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:53.601723 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:53.601987 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:53.602307 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:53.602310 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:53.602312 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:53.602405 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:53.602825 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:53.602830 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:53.602927 (+ 97us) client_negotiation.cc:769] Sending connection context
0811 11:55:53.603048 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 11:55:53.603196 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2549,"mutex_wait_us":659,"thread_start_us":78,"threads_started":1}
I20260811 11:55:53.603405 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.578121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:43821 (local address 127.6.143.188:43931)
0811 11:55:53.578270 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:53.578276 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:53.580745 (+ 2469us) server_negotiation.cc:407] Connection header received
0811 11:55:53.580879 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:53.580882 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:53.580940 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:53.581026 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:53.581935 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.582711 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.586969 (+ 4258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.587320 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.587417 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.599652 (+ 12235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:53.599656 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:53.599659 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:53.599692 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:53.601415 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:53.602111 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:53.602115 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:53.602117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:53.602205 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:53.602512 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:53.602515 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:53.602517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:53.602717 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:53.602807 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:53.603139 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 11:55:53.603256 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 11:55:53.604439 14725 heartbeater.cc:347] Connected to a master server at 127.6.143.188:43931
I20260811 11:55:53.604528 14725 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:53.604722 14725 heartbeater.cc:511] Master 127.6.143.188:43931 requested a full tablet report, sending...
I20260811 11:55:53.605453 14225 ts_manager.cc:194] Registered new tserver with Master: d8af8d74ff8b451e80081f46f94504a2 (127.6.143.131:42641)
I20260811 11:55:53.607492 14740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.579914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60858 (local address 127.6.143.188:43931)
0811 11:55:53.580977 (+ 1063us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:53.580982 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:53.582920 (+ 1938us) server_negotiation.cc:407] Connection header received
0811 11:55:53.582966 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:53.582970 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:53.583023 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:53.583118 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:53.583991 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.584792 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.585992 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.586846 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.586964 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.604123 (+ 17159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:53.604127 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:53.604129 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:53.604155 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:53.605972 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:53.606422 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:53.606425 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:53.606426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:53.606498 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:53.606762 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:53.606764 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:53.606765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:53.606933 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:53.607026 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:53.607240 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 11:55:53.607331 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":992,"thread_start_us":91,"threads_started":1}
I20260811 11:55:53.605631 13371 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51204:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:55:53.609426 13371 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:55:53.611791 14184 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8fedd24da8fe4ac59ffba44f177b1009" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: false } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: NON_VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: true } } }
I20260811 11:55:53.612001 14184 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:53.615399 14745 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009: attempting to promote NON_VOTER 868a991373ad4c90a1e2aa593bc50755 to VOTER
I20260811 11:55:53.615731 14745 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: false } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: false } }
I20260811 11:55:53.617278 14237 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [term 1 LEARNER]: Refusing update from remote peer 8fedd24da8fe4ac59ffba44f177b1009: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 11:55:53.617398 14032 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [term 1 FOLLOWER]: Refusing update from remote peer 8fedd24da8fe4ac59ffba44f177b1009: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 11:55:53.617825 14744 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [LEADER]: Connected to new peer: Peer: permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 11:55:53.618204 14744 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [LEADER]: Connected to new peer: Peer: permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 11:55:53.619712 14742 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 868a991373ad4c90a1e2aa593bc50755 (127.6.143.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: false } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: false } } }
I20260811 11:55:53.620338 14032 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 868a991373ad4c90a1e2aa593bc50755 (127.6.143.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: false } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: false } } }
I20260811 11:55:53.620369 14237 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 868a991373ad4c90a1e2aa593bc50755 (127.6.143.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: false } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: false } } }
I20260811 11:55:53.621222 14742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "8fedd24da8fe4ac59ffba44f177b1009" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: false } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: false } } }
I20260811 11:55:53.621330 14742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:53.621351 14744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8fedd24da8fe4ac59ffba44f177b1009" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: false } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: false } } }
I20260811 11:55:53.621441 14744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:53.621440 14742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "8fedd24da8fe4ac59ffba44f177b1009" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: false } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: false } } }
I20260811 11:55:53.621568 14742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:53.622251 14268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8fedd24da8fe4ac59ffba44f177b1009" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: false } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: false } } }
I20260811 11:55:53.622380 14268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:55:53.622679 14755 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "8fedd24da8fe4ac59ffba44f177b1009" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: false } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: false } } }
I20260811 11:55:53.622795 14755 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:55:53.631074 13969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.624665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:42641 (local address 127.0.0.1:43638)
0811 11:55:53.624952 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:53.624967 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:53.625046 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:53.626551 (+ 1505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:53.626555 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:53.626574 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:53.626694 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.626701 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.628350 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.628361 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.629512 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.629524 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.630701 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.630705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.630791 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.630817 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 11:55:53.630863 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 11:55:53.630926 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":24}
I20260811 11:55:53.631367 14758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.625003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:39527 (local address 127.0.0.1:32896)
0811 11:55:53.626095 (+ 1092us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:53.626108 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:53.626232 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:53.626960 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:53.626963 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:53.626983 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:53.627070 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.627077 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.628617 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.628622 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.629712 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.629723 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.630962 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.630965 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.631028 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.631064 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 11:55:53.631156 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 11:55:53.631226 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1033,"mutex_wait_us":86,"thread_start_us":97,"threads_started":1}
I20260811 11:55:53.631441 14760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.625749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32896 (local address 127.6.143.129:39527)
0811 11:55:53.626023 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:53.626038 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:53.626207 (+ 169us) server_negotiation.cc:407] Connection header received
0811 11:55:53.626350 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:53.626357 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:53.626635 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:53.626998 (+ 363us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:53.627170 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.628484 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.629837 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.630647 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.630753 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.630906 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:53.631193 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 11:55:53.631296 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":94,"threads_started":1}
I20260811 11:55:53.631974 14756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.625327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:42017 (local address 127.0.0.1:42484)
0811 11:55:53.626446 (+ 1119us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:53.626461 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:53.626615 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:53.627190 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:53.627193 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:53.627212 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:53.627285 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.627291 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.628733 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.628737 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.629903 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.629913 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.631530 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.631535 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.631641 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.631668 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 11:55:53.631750 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 11:55:53.631818 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1035,"mutex_wait_us":2,"spinlock_wait_cycles":768,"thread_start_us":109,"threads_started":1}
I20260811 11:55:53.632205 14759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.625513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42484 (local address 127.6.143.130:42017)
0811 11:55:53.625967 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:53.625976 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:53.626539 (+ 563us) server_negotiation.cc:407] Connection header received
0811 11:55:53.626785 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:53.626792 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:53.627014 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:53.627164 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:53.627421 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.628617 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.630407 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.631188 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.631304 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.631457 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:53.631794 (+ 337us) server_negotiation.cc:299] Negotiation successful
0811 11:55:53.631881 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":132,"threads_started":1}
I20260811 11:55:53.632452 14757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.625208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43638 (local address 127.6.143.131:42641)
0811 11:55:53.626115 (+ 907us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:53.626123 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:53.626142 (+ 19us) server_negotiation.cc:407] Connection header received
0811 11:55:53.626203 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:53.626207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:53.626370 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:53.626584 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:53.627322 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.628222 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.629640 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.630569 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.632021 (+ 1452us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.632170 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:53.632225 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 11:55:53.632318 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":830,"thread_start_us":106,"threads_started":1}
I20260811 11:55:53.634516 14353 tablet_service.cc:1467] Processing CreateTablet for tablet 9d22fcdd7f0b42f28356fe6a01532461 (DEFAULT_TABLE table=test-table [id=5025f11832cf4681a142475133f80d9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:53.634964 14502 tablet_service.cc:1467] Processing CreateTablet for tablet 9d22fcdd7f0b42f28356fe6a01532461 (DEFAULT_TABLE table=test-table [id=5025f11832cf4681a142475133f80d9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:53.634961 14652 tablet_service.cc:1467] Processing CreateTablet for tablet 9d22fcdd7f0b42f28356fe6a01532461 (DEFAULT_TABLE table=test-table [id=5025f11832cf4681a142475133f80d9f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:53.635502 14353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d22fcdd7f0b42f28356fe6a01532461. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:53.636008 14502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d22fcdd7f0b42f28356fe6a01532461. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:53.636008 14652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d22fcdd7f0b42f28356fe6a01532461. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:53.642504 14761 tablet_bootstrap.cc:492] T 9d22fcdd7f0b42f28356fe6a01532461 P b063e255083e4cdda907e958091e2606: Bootstrap starting.
I20260811 11:55:53.643708 14762 tablet_bootstrap.cc:492] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2: Bootstrap starting.
I20260811 11:55:53.645592 14761 tablet_bootstrap.cc:654] T 9d22fcdd7f0b42f28356fe6a01532461 P b063e255083e4cdda907e958091e2606: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:53.645903 14762 tablet_bootstrap.cc:654] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:53.646543 14761 log.cc:824] T 9d22fcdd7f0b42f28356fe6a01532461 P b063e255083e4cdda907e958091e2606: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:53.646620 14762 log.cc:824] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:53.647257 14763 tablet_bootstrap.cc:492] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe: Bootstrap starting.
I20260811 11:55:53.648842 14761 tablet_bootstrap.cc:492] T 9d22fcdd7f0b42f28356fe6a01532461 P b063e255083e4cdda907e958091e2606: No bootstrap required, opened a new log
I20260811 11:55:53.648865 14762 tablet_bootstrap.cc:492] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2: No bootstrap required, opened a new log
I20260811 11:55:53.649057 14761 ts_tablet_manager.cc:1397] T 9d22fcdd7f0b42f28356fe6a01532461 P b063e255083e4cdda907e958091e2606: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 11:55:53.649060 14762 ts_tablet_manager.cc:1397] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 11:55:53.649389 14763 tablet_bootstrap.cc:654] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:53.650229 14763 log.cc:824] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:53.652412 14763 tablet_bootstrap.cc:492] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe: No bootstrap required, opened a new log
I20260811 11:55:53.652637 14763 ts_tablet_manager.cc:1397] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 11:55:53.653357 14761 raft_consensus.cc:348] T 9d22fcdd7f0b42f28356fe6a01532461 P b063e255083e4cdda907e958091e2606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8af8d74ff8b451e80081f46f94504a2" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42641 } } peers { permanent_uuid: "b063e255083e4cdda907e958091e2606" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39527 } } peers { permanent_uuid: "0e664a94aff147c5aea4d0c580f610fe" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42017 } }
I20260811 11:55:53.653599 14761 raft_consensus.cc:374] T 9d22fcdd7f0b42f28356fe6a01532461 P b063e255083e4cdda907e958091e2606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:53.653663 14761 raft_consensus.cc:729] T 9d22fcdd7f0b42f28356fe6a01532461 P b063e255083e4cdda907e958091e2606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b063e255083e4cdda907e958091e2606, State: Initialized, Role: FOLLOWER
I20260811 11:55:53.653541 14762 raft_consensus.cc:348] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8af8d74ff8b451e80081f46f94504a2" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42641 } } peers { permanent_uuid: "b063e255083e4cdda907e958091e2606" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39527 } } peers { permanent_uuid: "0e664a94aff147c5aea4d0c580f610fe" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42017 } }
I20260811 11:55:53.653811 14762 raft_consensus.cc:374] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:53.653874 14762 raft_consensus.cc:729] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8af8d74ff8b451e80081f46f94504a2, State: Initialized, Role: FOLLOWER
I20260811 11:55:53.654208 14761 consensus_queue.cc:260] T 9d22fcdd7f0b42f28356fe6a01532461 P b063e255083e4cdda907e958091e2606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8af8d74ff8b451e80081f46f94504a2" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42641 } } peers { permanent_uuid: "b063e255083e4cdda907e958091e2606" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39527 } } peers { permanent_uuid: "0e664a94aff147c5aea4d0c580f610fe" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42017 } }
I20260811 11:55:53.654305 14762 consensus_queue.cc:260] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8af8d74ff8b451e80081f46f94504a2" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42641 } } peers { permanent_uuid: "b063e255083e4cdda907e958091e2606" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39527 } } peers { permanent_uuid: "0e664a94aff147c5aea4d0c580f610fe" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42017 } }
I20260811 11:55:53.654867 14423 heartbeater.cc:503] Master 127.6.143.190:46619 was elected leader, sending a full tablet report...
I20260811 11:55:53.655227 14761 ts_tablet_manager.cc:1428] T 9d22fcdd7f0b42f28356fe6a01532461 P b063e255083e4cdda907e958091e2606: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260811 11:55:53.655581 14762 ts_tablet_manager.cc:1428] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 11:55:53.655910 14724 heartbeater.cc:503] Master 127.6.143.190:46619 was elected leader, sending a full tablet report...
I20260811 11:55:53.659462 14763 raft_consensus.cc:348] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8af8d74ff8b451e80081f46f94504a2" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42641 } } peers { permanent_uuid: "b063e255083e4cdda907e958091e2606" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39527 } } peers { permanent_uuid: "0e664a94aff147c5aea4d0c580f610fe" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42017 } }
I20260811 11:55:53.659749 14763 raft_consensus.cc:374] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:53.659806 14763 raft_consensus.cc:729] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e664a94aff147c5aea4d0c580f610fe, State: Initialized, Role: FOLLOWER
I20260811 11:55:53.660252 14763 consensus_queue.cc:260] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8af8d74ff8b451e80081f46f94504a2" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42641 } } peers { permanent_uuid: "b063e255083e4cdda907e958091e2606" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39527 } } peers { permanent_uuid: "0e664a94aff147c5aea4d0c580f610fe" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42017 } }
I20260811 11:55:53.660871 14576 heartbeater.cc:503] Master 127.6.143.190:46619 was elected leader, sending a full tablet report...
I20260811 11:55:53.661226 14763 ts_tablet_manager.cc:1428] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
W20260811 11:55:53.826032 14727 tablet.cc:2369] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 11:55:53.860030 14581 tablet.cc:2369] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 11:55:53.905443 14427 tablet.cc:2369] T 9d22fcdd7f0b42f28356fe6a01532461 P b063e255083e4cdda907e958091e2606: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:55:53.913383 14770 raft_consensus.cc:482] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:55:53.913659 14770 raft_consensus.cc:504] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8af8d74ff8b451e80081f46f94504a2" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42641 } } peers { permanent_uuid: "b063e255083e4cdda907e958091e2606" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39527 } } peers { permanent_uuid: "0e664a94aff147c5aea4d0c580f610fe" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42017 } }
I20260811 11:55:53.915045 14770 leader_election.cc:288] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8af8d74ff8b451e80081f46f94504a2 (127.6.143.131:42641), b063e255083e4cdda907e958091e2606 (127.6.143.129:39527)
I20260811 11:55:53.917327 14769 raft_consensus.cc:482] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:55:53.917512 14769 raft_consensus.cc:504] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8af8d74ff8b451e80081f46f94504a2" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42641 } } peers { permanent_uuid: "b063e255083e4cdda907e958091e2606" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39527 } } peers { permanent_uuid: "0e664a94aff147c5aea4d0c580f610fe" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42017 } }
I20260811 11:55:53.918677 14769 leader_election.cc:288] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b063e255083e4cdda907e958091e2606 (127.6.143.129:39527), 0e664a94aff147c5aea4d0c580f610fe (127.6.143.130:42017)
I20260811 11:55:53.919798 14760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.915393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:58123 (local address 127.6.143.129:39527)
0811 11:55:53.915578 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:53.915583 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:53.915691 (+ 108us) server_negotiation.cc:407] Connection header received
0811 11:55:53.915818 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:53.915821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:53.915864 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:53.915990 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:53.916356 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.916950 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.918495 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.919088 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.919177 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.919224 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:53.919610 (+ 386us) server_negotiation.cc:299] Negotiation successful
0811 11:55:53.919686 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260811 11:55:53.919812 14771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.915302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:39527 (local address 127.6.143.130:58123)
0811 11:55:53.915585 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:53.915599 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:53.915723 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:53.916054 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:53.916058 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:53.916082 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:53.916211 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.916216 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.917063 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.917068 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.918331 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.918343 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.919331 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.919334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.919487 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.919506 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 11:55:53.919590 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 11:55:53.919669 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":219,"mutex_wait_us":30,"thread_start_us":82,"threads_started":1}
I20260811 11:55:53.919924 14757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.915438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:56909 (local address 127.6.143.131:42641)
0811 11:55:53.915688 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:53.915692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:53.916028 (+ 336us) server_negotiation.cc:407] Connection header received
0811 11:55:53.916158 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:53.916161 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:53.916231 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:53.916345 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:53.916616 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.917275 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.918472 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.919053 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.919151 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.919210 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:53.919707 (+ 497us) server_negotiation.cc:299] Negotiation successful
0811 11:55:53.919785 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165}
I20260811 11:55:53.920430 14374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d22fcdd7f0b42f28356fe6a01532461" candidate_uuid: "0e664a94aff147c5aea4d0c580f610fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b063e255083e4cdda907e958091e2606" is_pre_election: true
I20260811 11:55:53.920697 14772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.915303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:42641 (local address 127.6.143.130:56909)
0811 11:55:53.915960 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:53.915974 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:53.916062 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:53.916356 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:53.916360 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:53.916376 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:53.916482 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.916488 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.917395 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.917402 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.918315 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.918325 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.919410 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.919413 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.919558 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.919591 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 11:55:53.920423 (+ 832us) client_negotiation.cc:240] Negotiation successful
0811 11:55:53.920502 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"mutex_wait_us":76,"thread_start_us":69,"threads_started":1}
I20260811 11:55:53.920722 14374 raft_consensus.cc:2454] T 9d22fcdd7f0b42f28356fe6a01532461 P b063e255083e4cdda907e958091e2606 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e664a94aff147c5aea4d0c580f610fe in term 0.
I20260811 11:55:53.921222 14456 leader_election.cc:302] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e664a94aff147c5aea4d0c580f610fe, b063e255083e4cdda907e958091e2606; no voters:
I20260811 11:55:53.921396 14672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d22fcdd7f0b42f28356fe6a01532461" candidate_uuid: "0e664a94aff147c5aea4d0c580f610fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8af8d74ff8b451e80081f46f94504a2" is_pre_election: true
I20260811 11:55:53.921559 14770 raft_consensus.cc:2781] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:55:53.921579 14672 raft_consensus.cc:2454] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e664a94aff147c5aea4d0c580f610fe in term 0.
I20260811 11:55:53.921693 14770 raft_consensus.cc:482] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:55:53.921934 14770 raft_consensus.cc:3037] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:53.923314 14730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.918842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:39527 (local address 127.6.143.131:34925)
0811 11:55:53.918991 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:53.919005 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:53.919079 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:53.920203 (+ 1124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:53.920206 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:53.920222 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:53.920324 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.920330 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.920966 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.920969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.922111 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.922124 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.922894 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.922898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.923005 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.923031 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 11:55:53.923117 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 11:55:53.923186 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 11:55:53.923324 14760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.919831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:34925 (local address 127.6.143.129:39527)
0811 11:55:53.920013 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:53.920017 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:53.920030 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:55:53.920076 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:53.920077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:53.920108 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:53.920192 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:53.920414 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.920844 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.922241 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.922748 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.922886 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.922930 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:53.923151 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 11:55:53.923211 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":89,"threads_started":1}
I20260811 11:55:53.923285 14770 raft_consensus.cc:504] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8af8d74ff8b451e80081f46f94504a2" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42641 } } peers { permanent_uuid: "b063e255083e4cdda907e958091e2606" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39527 } } peers { permanent_uuid: "0e664a94aff147c5aea4d0c580f610fe" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42017 } }
I20260811 11:55:53.923815 14374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d22fcdd7f0b42f28356fe6a01532461" candidate_uuid: "d8af8d74ff8b451e80081f46f94504a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b063e255083e4cdda907e958091e2606" is_pre_election: true
I20260811 11:55:53.924011 14770 leader_election.cc:288] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe [CANDIDATE]: Term 1 election: Requested vote from peers d8af8d74ff8b451e80081f46f94504a2 (127.6.143.131:42641), b063e255083e4cdda907e958091e2606 (127.6.143.129:39527)
I20260811 11:55:53.924010 14374 raft_consensus.cc:2454] T 9d22fcdd7f0b42f28356fe6a01532461 P b063e255083e4cdda907e958091e2606 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8af8d74ff8b451e80081f46f94504a2 in term 0.
I20260811 11:55:53.924324 14604 leader_election.cc:302] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b063e255083e4cdda907e958091e2606, d8af8d74ff8b451e80081f46f94504a2; no voters:
I20260811 11:55:53.924538 14374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d22fcdd7f0b42f28356fe6a01532461" candidate_uuid: "0e664a94aff147c5aea4d0c580f610fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b063e255083e4cdda907e958091e2606"
I20260811 11:55:53.924561 14769 raft_consensus.cc:2781] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:55:53.924638 14769 raft_consensus.cc:482] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:55:53.924695 14769 raft_consensus.cc:3037] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:53.924660 14374 raft_consensus.cc:3037] T 9d22fcdd7f0b42f28356fe6a01532461 P b063e255083e4cdda907e958091e2606 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:53.924939 14672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d22fcdd7f0b42f28356fe6a01532461" candidate_uuid: "0e664a94aff147c5aea4d0c580f610fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8af8d74ff8b451e80081f46f94504a2"
I20260811 11:55:53.925535 14729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.919095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:42017 (local address 127.6.143.131:51589)
0811 11:55:53.919243 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:53.919255 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:53.919325 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:53.921538 (+ 2213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:53.921541 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:53.921560 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:53.921661 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.921667 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.922594 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.922597 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.923701 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:53.923712 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.925165 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.925169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:53.925269 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.925290 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 11:55:53.925371 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 11:55:53.925435 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260811 11:55:53.925767 14769 raft_consensus.cc:504] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8af8d74ff8b451e80081f46f94504a2" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42641 } } peers { permanent_uuid: "b063e255083e4cdda907e958091e2606" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39527 } } peers { permanent_uuid: "0e664a94aff147c5aea4d0c580f610fe" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42017 } }
I20260811 11:55:53.925988 14759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.919359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:51589 (local address 127.6.143.130:42017)
0811 11:55:53.919471 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:53.919475 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:53.919486 (+ 11us) server_negotiation.cc:407] Connection header received
0811 11:55:53.919518 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:53.919521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:53.919559 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:53.919624 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:53.921827 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.922476 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.923816 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.924406 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.924512 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.924560 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:53.925775 (+ 1215us) server_negotiation.cc:299] Negotiation successful
0811 11:55:53.925846 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 11:55:53.926026 14672 raft_consensus.cc:2379] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 0e664a94aff147c5aea4d0c580f610fe in current term 1: Already voted for candidate d8af8d74ff8b451e80081f46f94504a2 in this term.
I20260811 11:55:53.926047 14374 raft_consensus.cc:2454] T 9d22fcdd7f0b42f28356fe6a01532461 P b063e255083e4cdda907e958091e2606 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e664a94aff147c5aea4d0c580f610fe in term 1.
I20260811 11:55:53.926128 14769 leader_election.cc:288] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2 [CANDIDATE]: Term 1 election: Requested vote from peers b063e255083e4cdda907e958091e2606 (127.6.143.129:39527), 0e664a94aff147c5aea4d0c580f610fe (127.6.143.130:42017)
I20260811 11:55:53.926518 14523 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d22fcdd7f0b42f28356fe6a01532461" candidate_uuid: "d8af8d74ff8b451e80081f46f94504a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e664a94aff147c5aea4d0c580f610fe" is_pre_election: true
I20260811 11:55:53.926614 14374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d22fcdd7f0b42f28356fe6a01532461" candidate_uuid: "d8af8d74ff8b451e80081f46f94504a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b063e255083e4cdda907e958091e2606"
I20260811 11:55:53.926744 14523 raft_consensus.cc:2379] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d8af8d74ff8b451e80081f46f94504a2 in current term 1: Already voted for candidate 0e664a94aff147c5aea4d0c580f610fe in this term.
I20260811 11:55:53.926776 14374 raft_consensus.cc:2379] T 9d22fcdd7f0b42f28356fe6a01532461 P b063e255083e4cdda907e958091e2606 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d8af8d74ff8b451e80081f46f94504a2 in current term 1: Already voted for candidate 0e664a94aff147c5aea4d0c580f610fe in this term.
I20260811 11:55:53.927147 14522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d22fcdd7f0b42f28356fe6a01532461" candidate_uuid: "d8af8d74ff8b451e80081f46f94504a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0e664a94aff147c5aea4d0c580f610fe"
I20260811 11:55:53.927286 14522 raft_consensus.cc:2379] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate d8af8d74ff8b451e80081f46f94504a2 in current term 1: Already voted for candidate 0e664a94aff147c5aea4d0c580f610fe in this term.
I20260811 11:55:53.927505 14456 leader_election.cc:302] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0e664a94aff147c5aea4d0c580f610fe, b063e255083e4cdda907e958091e2606; no voters: d8af8d74ff8b451e80081f46f94504a2
I20260811 11:55:53.927516 14604 leader_election.cc:302] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d8af8d74ff8b451e80081f46f94504a2; no voters: 0e664a94aff147c5aea4d0c580f610fe, b063e255083e4cdda907e958091e2606
I20260811 11:55:53.927711 14769 raft_consensus.cc:2726] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260811 11:55:53.927747 14770 raft_consensus.cc:2781] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:53.927958 14770 raft_consensus.cc:686] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe [term 1 LEADER]: Becoming Leader. State: Replica: 0e664a94aff147c5aea4d0c580f610fe, State: Running, Role: LEADER
I20260811 11:55:53.928288 14770 consensus_queue.cc:237] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8af8d74ff8b451e80081f46f94504a2" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42641 } } peers { permanent_uuid: "b063e255083e4cdda907e958091e2606" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39527 } } peers { permanent_uuid: "0e664a94aff147c5aea4d0c580f610fe" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42017 } }
I20260811 11:55:53.931859 13371 catalog_manager.cc:5495] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e664a94aff147c5aea4d0c580f610fe (127.6.143.130). New cstate: current_term: 1 leader_uuid: "0e664a94aff147c5aea4d0c580f610fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8af8d74ff8b451e80081f46f94504a2" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b063e255083e4cdda907e958091e2606" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e664a94aff147c5aea4d0c580f610fe" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42017 } health_report { overall_health: HEALTHY } } }
I20260811 11:55:53.990859 14759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:53.987483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42492 (local address 127.6.143.130:42017)
0811 11:55:53.987622 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:53.987629 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:53.987647 (+ 18us) server_negotiation.cc:407] Connection header received
0811 11:55:53.987726 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:53.987728 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:53.987802 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:53.987918 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:55:53.988196 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.988689 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.989506 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:53.989709 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:53.989789 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:53.990068 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:55:53.990496 (+ 428us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:55:53.990588 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:53.990643 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 11:55:53.990723 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 11:55:53.996721 14672 raft_consensus.cc:1261] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2 [term 1 FOLLOWER]: Refusing update from remote peer 0e664a94aff147c5aea4d0c580f610fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:53.996722 14374 raft_consensus.cc:1261] T 9d22fcdd7f0b42f28356fe6a01532461 P b063e255083e4cdda907e958091e2606 [term 1 FOLLOWER]: Refusing update from remote peer 0e664a94aff147c5aea4d0c580f610fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:53.997375 14774 consensus_queue.cc:1035] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8af8d74ff8b451e80081f46f94504a2" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:53.997586 14770 consensus_queue.cc:1035] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b063e255083e4cdda907e958091e2606" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:54.006693 14778 mvcc.cc:205] Tried to move back new op lower bound from 7317296553963474944 to 7317296553693073408. Current Snapshot: MvccSnapshot[applied={T|T < 7317296553963474944}]
I20260811 11:55:54.006806 14780 mvcc.cc:205] Tried to move back new op lower bound from 7317296553963474944 to 7317296553693073408. Current Snapshot: MvccSnapshot[applied={T|T < 7317296553963474944}]
I20260811 11:55:54.007390 14779 mvcc.cc:205] Tried to move back new op lower bound from 7317296553963474944 to 7317296553693073408. Current Snapshot: MvccSnapshot[applied={T|T < 7317296553963474944}]
I20260811 11:55:54.015445 13370 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:55:54.015630 13370 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51204:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:55:54.019480 13370 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8fedd24da8fe4ac59ffba44f177b1009: Sending DeleteTablet for 3 replicas of tablet 9d22fcdd7f0b42f28356fe6a01532461
I20260811 11:55:54.020308 14502 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d22fcdd7f0b42f28356fe6a01532461 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:54 UTC) from {username='oryx', principal='oryx/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:42484
I20260811 11:55:54.020305 14353 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d22fcdd7f0b42f28356fe6a01532461 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:54 UTC) from {username='oryx', principal='oryx/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:32896
I20260811 11:55:54.021063 14652 tablet_service.cc:1514] Processing DeleteTablet for tablet 9d22fcdd7f0b42f28356fe6a01532461 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:54 UTC) from {username='oryx', principal='oryx/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:43638
I20260811 11:55:54.021548 14791 tablet_replica.cc:330] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2: stopping tablet replica
I20260811 11:55:54.021562 14789 tablet_replica.cc:330] T 9d22fcdd7f0b42f28356fe6a01532461 P b063e255083e4cdda907e958091e2606: stopping tablet replica
I20260811 11:55:54.021852 14789 raft_consensus.cc:2229] T 9d22fcdd7f0b42f28356fe6a01532461 P b063e255083e4cdda907e958091e2606 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:55:54.021852 14791 raft_consensus.cc:2229] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:55:54.022173 14791 raft_consensus.cc:2258] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:54.022171 14789 raft_consensus.cc:2258] T 9d22fcdd7f0b42f28356fe6a01532461 P b063e255083e4cdda907e958091e2606 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:54.023746 14789 ts_tablet_manager.cc:1905] T 9d22fcdd7f0b42f28356fe6a01532461 P b063e255083e4cdda907e958091e2606: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:54.023746 14791 ts_tablet_manager.cc:1905] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:54.023970 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 13337
I20260811 11:55:54.025936 14790 tablet_replica.cc:330] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe: stopping tablet replica
I20260811 11:55:54.026333 14790 raft_consensus.cc:2229] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:55:54.027271 14790 raft_consensus.cc:2258] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:54.027843 14791 ts_tablet_manager.cc:1918] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:55:54.028044 14791 log.cc:1196] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/wal/wals/9d22fcdd7f0b42f28356fe6a01532461
I20260811 11:55:54.028472 14791 ts_tablet_manager.cc:1939] T 9d22fcdd7f0b42f28356fe6a01532461 P d8af8d74ff8b451e80081f46f94504a2: Deleting consensus metadata
I20260811 11:55:54.029181 14790 ts_tablet_manager.cc:1905] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:54.031296 14789 ts_tablet_manager.cc:1918] T 9d22fcdd7f0b42f28356fe6a01532461 P b063e255083e4cdda907e958091e2606: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:55:54.031426 14789 log.cc:1196] T 9d22fcdd7f0b42f28356fe6a01532461 P b063e255083e4cdda907e958091e2606: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-0/wal/wals/9d22fcdd7f0b42f28356fe6a01532461
I20260811 11:55:54.031720 14789 ts_tablet_manager.cc:1939] T 9d22fcdd7f0b42f28356fe6a01532461 P b063e255083e4cdda907e958091e2606: Deleting consensus metadata
I20260811 11:55:54.035127 14790 ts_tablet_manager.cc:1918] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:55:54.035235 14790 log.cc:1196] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/wal/wals/9d22fcdd7f0b42f28356fe6a01532461
I20260811 11:55:54.035573 14790 ts_tablet_manager.cc:1939] T 9d22fcdd7f0b42f28356fe6a01532461 P 0e664a94aff147c5aea4d0c580f610fe: Deleting consensus metadata
W20260811 11:55:54.035921 14303 connection.cc:535] server connection from 127.0.0.1:32896 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 11:55:54.036625 14603 connection.cc:535] server connection from 127.0.0.1:43638 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 11:55:54.038183 14454 connection.cc:441] server connection from 127.0.0.1:42484 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:42484 (request call id 1) could send its response
I20260811 11:55:54.043303 14738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:54.022817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38440 (local address 127.6.143.189:43751)
0811 11:55:54.022928 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:54.022935 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:54.032221 (+ 9286us) server_negotiation.cc:407] Connection header received
0811 11:55:54.032431 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:54.032435 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:54.032493 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:54.032549 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:54.033619 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:54.034404 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:54.035771 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:54.036143 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:54.036236 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:54.038794 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:54.038798 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:54.038801 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:54.038828 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:54.040471 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:54.041480 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:54.041487 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:54.041492 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:54.041584 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:54.041921 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:54.041928 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:54.041933 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:54.042183 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:54.042313 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:54.043020 (+ 707us) server_negotiation.cc:299] Negotiation successful
0811 11:55:54.043131 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 11:55:54.050288 14159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:54.041072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38446 (local address 127.6.143.189:43751)
0811 11:55:54.041183 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:54.041186 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:54.041350 (+ 164us) server_negotiation.cc:407] Connection header received
0811 11:55:54.041512 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:54.041515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:54.041560 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:54.041614 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:54.042493 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:54.043187 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:54.044568 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:54.044963 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:54.045043 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:54.046931 (+ 1888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:54.046934 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:54.046937 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:54.046960 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:54.048691 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:54.049126 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:54.049130 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:54.049132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:54.049210 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:54.049486 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:54.049490 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:54.049491 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:54.049649 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:54.049779 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:54.050009 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 11:55:54.050113 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 11:55:54.053031 14433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:54.040720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60868 (local address 127.6.143.188:43931)
0811 11:55:54.040832 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:54.040836 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:54.040848 (+ 12us) server_negotiation.cc:407] Connection header received
0811 11:55:54.040903 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:54.040906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:54.040951 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:54.041025 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:54.042206 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:54.042992 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:54.044409 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:54.044782 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:54.044886 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:54.046632 (+ 1746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:54.046636 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:54.046639 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:54.046663 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:54.050016 (+ 3353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:54.051368 (+ 1352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:54.051372 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:54.051373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:54.051499 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:54.051800 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:54.051803 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:54.051805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:54.052048 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:54.052169 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:54.052717 (+ 548us) server_negotiation.cc:299] Negotiation successful
0811 11:55:54.052881 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":24,"server-negotiator.queue_time_us":42}
I20260811 11:55:54.054831 14740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:54.022711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60860 (local address 127.6.143.188:43931)
0811 11:55:54.022822 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:54.022826 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:54.042921 (+ 20095us) server_negotiation.cc:407] Connection header received
0811 11:55:54.044283 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:54.044286 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:54.044325 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:54.044369 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:54.045326 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:54.046151 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:54.047274 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:54.047510 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:54.047623 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:54.051355 (+ 3732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:54.051359 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:54.051362 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:54.051399 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:54.053108 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:54.053574 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:54.053578 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:54.053580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:54.053690 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:54.054045 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:54.054048 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:54.054050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:54.054247 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:54.054370 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:54.054563 (+ 193us) server_negotiation.cc:299] Negotiation successful
0811 11:55:54.054675 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":513,"server-negotiator.queue_time_us":30,"spinlock_wait_cycles":9600}
I20260811 11:55:55.009074 14799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.008578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.6.143.130:36937)
0811 11:55:55.008848 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:55.008942 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":85,"threads_started":1}
W20260811 11:55:55.009572 14576 heartbeater.cc:644] Failed to heartbeat to 127.6.143.190:46619 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
I20260811 11:55:55.010079 14800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.009671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.6.143.131:51119)
0811 11:55:55.009901 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:55.009987 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":79,"threads_started":1}
W20260811 11:55:55.010550 14724 heartbeater.cc:644] Failed to heartbeat to 127.6.143.190:46619 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
I20260811 11:55:55.012672 14801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.012182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.6.143.129:60829)
0811 11:55:55.012440 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:55.012531 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":67,"threads_started":1}
W20260811 11:55:55.013003 14423 heartbeater.cc:644] Failed to heartbeat to 127.6.143.190:46619 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
I20260811 11:55:55.519862 14802 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8fedd24da8fe4ac59ffba44f177b1009)
I20260811 11:55:55.520084 14802 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: false } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: false } }
I20260811 11:55:55.520859 14803 raft_consensus.cc:482] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 8fedd24da8fe4ac59ffba44f177b1009)
I20260811 11:55:55.521073 14803 raft_consensus.cc:504] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: false } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: false } }
I20260811 11:55:55.521181 14802 leader_election.cc:288] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fedd24da8fe4ac59ffba44f177b1009 (127.6.143.190:46619), 868a991373ad4c90a1e2aa593bc50755 (127.6.143.188:43931)
I20260811 11:55:55.521786 14804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.521345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.0.0.1:51648)
0811 11:55:55.521564 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:55.521659 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":81,"threads_started":1}
W20260811 11:55:55.522101 14006 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
I20260811 11:55:55.522318 14803 leader_election.cc:288] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8fedd24da8fe4ac59ffba44f177b1009 (127.6.143.190:46619), 79d3c932590e456c86fee8a8faecbec3 (127.6.143.189:43751)
I20260811 11:55:55.522701 14808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.522291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.0.0.1:51654)
0811 11:55:55.522520 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:55.522594 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":80,"threads_started":1}
W20260811 11:55:55.522866 14211 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
I20260811 11:55:55.523743 14804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.523524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.0.0.1:51658)
0811 11:55:55.523594 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:55.523652 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260811 11:55:55.523844 14809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.523515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.0.0.1:51656)
0811 11:55:55.523713 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:55.523768 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
W20260811 11:55:55.523872 14006 leader_election.cc:334] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8fedd24da8fe4ac59ffba44f177b1009 (127.6.143.190:46619): Network error: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
W20260811 11:55:55.523970 14211 leader_election.cc:334] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 8fedd24da8fe4ac59ffba44f177b1009 (127.6.143.190:46619): Network error: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
I20260811 11:55:55.525938 14806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.521894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60874 (local address 127.6.143.188:43931)
0811 11:55:55.522095 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:55.522100 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:55.522117 (+ 17us) server_negotiation.cc:407] Connection header received
0811 11:55:55.522172 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:55.522177 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:55.522228 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:55.522327 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:55.522740 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:55.523606 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:55.524981 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:55.525469 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:55.525549 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:55.525597 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:55.525803 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 11:55:55.525861 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":55,"threads_started":1}
I20260811 11:55:55.525964 14805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.521676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:43931 (local address 127.0.0.1:60874)
0811 11:55:55.521944 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:55.521959 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:55.522056 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:55.522464 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:55.522467 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:55.522484 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:55.522588 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:55.522594 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:55.523708 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:55.523721 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:55.524847 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:55.524857 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:55.525586 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:55.525590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:55.525677 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:55.525704 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 11:55:55.525783 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 11:55:55.525848 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":3,"thread_start_us":90,"threads_started":1}
I20260811 11:55:55.526472 14237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "79d3c932590e456c86fee8a8faecbec3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "868a991373ad4c90a1e2aa593bc50755" is_pre_election: true
I20260811 11:55:55.526700 14237 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79d3c932590e456c86fee8a8faecbec3 in term 1.
I20260811 11:55:55.527097 14006 leader_election.cc:302] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 79d3c932590e456c86fee8a8faecbec3, 868a991373ad4c90a1e2aa593bc50755; no voters: 8fedd24da8fe4ac59ffba44f177b1009
I20260811 11:55:55.527359 14802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 11:55:55.527452 14802 raft_consensus.cc:482] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 8fedd24da8fe4ac59ffba44f177b1009)
I20260811 11:55:55.527508 14802 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [term 1 FOLLOWER]: Advancing to term 2
I20260811 11:55:55.527601 14808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.522291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:43751 (local address 127.0.0.1:38460)
0811 11:55:55.522840 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:55.522853 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:55.522953 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:55.524215 (+ 1262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:55.524218 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:55.524239 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:55.524325 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:55.524330 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:55.525091 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:55.525094 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:55.526346 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:55.526359 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:55.527121 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:55.527125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:55.527267 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:55.527291 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 11:55:55.527373 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 11:55:55.527452 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"mutex_wait_us":268,"thread_start_us":75,"threads_started":1}
I20260811 11:55:55.527704 14810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.522594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38460 (local address 127.6.143.189:43751)
0811 11:55:55.523443 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:55.523453 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:55.523466 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:55:55.523505 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:55.523507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:55.524108 (+ 601us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:55.524197 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:55.524431 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:55.524981 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:55.526451 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:55.526996 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:55.527109 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:55.527161 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:55.527406 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 11:55:55.527556 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":783,"thread_start_us":113,"threads_started":1}
I20260811 11:55:55.528105 14032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "868a991373ad4c90a1e2aa593bc50755" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "79d3c932590e456c86fee8a8faecbec3" is_pre_election: true
I20260811 11:55:55.528781 14802 raft_consensus.cc:504] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: false } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: false } }
I20260811 11:55:55.529021 14032 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 868a991373ad4c90a1e2aa593bc50755 in current term 2: Already voted for candidate 79d3c932590e456c86fee8a8faecbec3 in this term.
I20260811 11:55:55.529140 14802 leader_election.cc:288] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [CANDIDATE]: Term 2 election: Requested vote from peers 8fedd24da8fe4ac59ffba44f177b1009 (127.6.143.190:46619), 868a991373ad4c90a1e2aa593bc50755 (127.6.143.188:43931)
I20260811 11:55:55.529469 14805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.529273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.0.0.1:51672)
0811 11:55:55.529344 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:55.529385 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260811 11:55:55.529647 14237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "79d3c932590e456c86fee8a8faecbec3" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "868a991373ad4c90a1e2aa593bc50755"
I20260811 11:55:55.529676 14212 leader_election.cc:302] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 868a991373ad4c90a1e2aa593bc50755; no voters: 79d3c932590e456c86fee8a8faecbec3, 8fedd24da8fe4ac59ffba44f177b1009
I20260811 11:55:55.529758 14237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [term 1 FOLLOWER]: Advancing to term 2
I20260811 11:55:55.530200 14805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.529980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.0.0.1:51680)
0811 11:55:55.530089 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:55.530138 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 11:55:55.530296 14006 leader_election.cc:334] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 8fedd24da8fe4ac59ffba44f177b1009 (127.6.143.190:46619): Network error: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
I20260811 11:55:55.530915 14237 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79d3c932590e456c86fee8a8faecbec3 in term 2.
I20260811 11:55:55.531018 14803 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 11:55:55.531241 14006 leader_election.cc:302] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 79d3c932590e456c86fee8a8faecbec3, 868a991373ad4c90a1e2aa593bc50755; no voters: 8fedd24da8fe4ac59ffba44f177b1009
I20260811 11:55:55.531517 14802 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 11:55:55.531709 14802 raft_consensus.cc:686] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [term 2 LEADER]: Becoming Leader. State: Replica: 79d3c932590e456c86fee8a8faecbec3, State: Running, Role: LEADER
I20260811 11:55:55.532009 14802 consensus_queue.cc:237] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: false } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: false } }
I20260811 11:55:55.532979 14813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79d3c932590e456c86fee8a8faecbec3. Latest consensus state: current_term: 2 leader_uuid: "79d3c932590e456c86fee8a8faecbec3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: false } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: false } } }
I20260811 11:55:55.533100 14813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:55.533383 14815 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:55:55.535506 14815 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:55:55.535972 14815 catalog_manager.cc:1259] Loaded cluster ID: e3111c441033471b8db261e6829e1e9e
I20260811 11:55:55.536018 14815 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:55:55.536566 14815 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:55:55.536962 14815 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3: Loaded TSK: 0
I20260811 11:55:55.537616 14815 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:55:55.563251 14022 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38446:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:55:55.566282 14022 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:55:55.569250 14805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.569039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.0.0.1:51698)
0811 11:55:55.569141 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:55.569188 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 11:55:55.569509 14237 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [term 2 FOLLOWER]: Refusing update from remote peer 79d3c932590e456c86fee8a8faecbec3: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260811 11:55:55.570017 14805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.569692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.0.0.1:51714)
0811 11:55:55.569899 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:55.569952 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143}
I20260811 11:55:55.570005 14813 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260811 11:55:55.570269 14006 consensus_peers.cc:487] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 -> Peer 8fedd24da8fe4ac59ffba44f177b1009 (127.6.143.190:46619): Couldn't send request to peer 8fedd24da8fe4ac59ffba44f177b1009. Status: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 11:55:55.571376 14803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 79d3c932590e456c86fee8a8faecbec3. Latest consensus state: current_term: 2 leader_uuid: "79d3c932590e456c86fee8a8faecbec3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: false } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: false } } }
I20260811 11:55:55.571514 14803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:55:55.572734 14802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "79d3c932590e456c86fee8a8faecbec3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: false } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: false } } }
I20260811 11:55:55.572875 14802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:55.573282 14803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "79d3c932590e456c86fee8a8faecbec3" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "8fedd24da8fe4ac59ffba44f177b1009" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46619 } } peers { permanent_uuid: "79d3c932590e456c86fee8a8faecbec3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 43751 } attrs { promote: false } } peers { permanent_uuid: "868a991373ad4c90a1e2aa593bc50755" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 43931 } attrs { promote: false } } }
I20260811 11:55:55.573405 14803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 868a991373ad4c90a1e2aa593bc50755 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:55:55.573519 14818 mvcc.cc:205] Tried to move back new op lower bound from 7317296560407093248 to 7317296560261267456. Current Snapshot: MvccSnapshot[applied={T|T < 7317296560407093248 or (T in {7317296560407093248})}]
I20260811 11:55:55.582744 14805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.578732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:42641 (local address 127.0.0.1:43642)
0811 11:55:55.578820 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:55.578835 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:55.578915 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:55.579306 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:55.579309 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:55.579325 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:55.579420 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:55.579426 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:55.580436 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:55.580439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:55.581601 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:55.581615 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:55.582382 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:55.582386 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:55.582471 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:55.582495 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 11:55:55.582589 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 11:55:55.582657 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":4}
I20260811 11:55:55.583134 14822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.578830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43642 (local address 127.6.143.131:42641)
0811 11:55:55.579110 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:55.579114 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:55.579125 (+ 11us) server_negotiation.cc:407] Connection header received
0811 11:55:55.579165 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:55.579167 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:55.579200 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:55.579297 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:55.579570 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:55.580304 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:55.581764 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:55.582260 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:55.582374 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:55.582443 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:55.582941 (+ 498us) server_negotiation.cc:299] Negotiation successful
0811 11:55:55.583016 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":105,"threads_started":1}
I20260811 11:55:55.583674 14652 tablet_service.cc:1467] Processing CreateTablet for tablet 8db6253f6f624f36a4a76af715b9bb73 (DEFAULT_TABLE table=test-table [id=474b20dce8794e9fb649efe4917837fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:55.584013 14652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8db6253f6f624f36a4a76af715b9bb73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:55.584731 14824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.579293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:39527 (local address 127.0.0.1:32908)
0811 11:55:55.579722 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:55.579734 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:55.579817 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:55.580390 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:55.580396 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:55.580418 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:55.580500 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:55.580505 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:55.581554 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:55.581557 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:55.582657 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:55.582667 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:55.584323 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:55.584330 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:55.584462 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:55.584493 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 11:55:55.584547 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 11:55:55.584613 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":20}
I20260811 11:55:55.585148 14804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.579505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:42017 (local address 127.0.0.1:42496)
0811 11:55:55.579722 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:55.579733 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:55.579799 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:55.580500 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:55.580504 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:55.580526 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:55.581829 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:55.581835 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:55.582828 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:55.582834 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:55.583937 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:55.583950 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:55.584741 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:55.584745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:55.584834 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:55.584854 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 11:55:55.584952 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 11:55:55.585025 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":1193,"thread_start_us":96,"threads_started":1}
I20260811 11:55:55.585162 14825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.579995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42496 (local address 127.6.143.130:42017)
0811 11:55:55.580217 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:55.580220 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:55.580233 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:55:55.580314 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:55.580317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:55.580366 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:55.580576 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:55.581957 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:55.582721 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:55.584113 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:55.584615 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:55.584742 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:55.584805 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:55.584969 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 11:55:55.585045 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":79,"threads_started":1}
I20260811 11:55:55.585832 14823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.579405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32908 (local address 127.6.143.129:39527)
0811 11:55:55.580156 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:55.580160 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:55.580174 (+ 14us) server_negotiation.cc:407] Connection header received
0811 11:55:55.580221 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:55.580223 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:55.580263 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:55.580602 (+ 339us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:55.580713 (+ 111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:55.581438 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:55.582907 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:55.584186 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:55.585129 (+ 943us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:55.585553 (+ 424us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:55.585604 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 11:55:55.585705 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":74,"threads_started":1}
I20260811 11:55:55.585865 14502 tablet_service.cc:1467] Processing CreateTablet for tablet 8db6253f6f624f36a4a76af715b9bb73 (DEFAULT_TABLE table=test-table [id=474b20dce8794e9fb649efe4917837fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:55.586131 14502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8db6253f6f624f36a4a76af715b9bb73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:55.586501 14353 tablet_service.cc:1467] Processing CreateTablet for tablet 8db6253f6f624f36a4a76af715b9bb73 (DEFAULT_TABLE table=test-table [id=474b20dce8794e9fb649efe4917837fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:55.586776 14353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8db6253f6f624f36a4a76af715b9bb73. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:55.588194 14826 tablet_bootstrap.cc:492] T 8db6253f6f624f36a4a76af715b9bb73 P d8af8d74ff8b451e80081f46f94504a2: Bootstrap starting.
I20260811 11:55:55.588968 14828 tablet_bootstrap.cc:492] T 8db6253f6f624f36a4a76af715b9bb73 P b063e255083e4cdda907e958091e2606: Bootstrap starting.
I20260811 11:55:55.589083 14827 tablet_bootstrap.cc:492] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe: Bootstrap starting.
I20260811 11:55:55.589249 14826 tablet_bootstrap.cc:654] T 8db6253f6f624f36a4a76af715b9bb73 P d8af8d74ff8b451e80081f46f94504a2: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:55.589933 14827 tablet_bootstrap.cc:654] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:55.589949 14828 tablet_bootstrap.cc:654] T 8db6253f6f624f36a4a76af715b9bb73 P b063e255083e4cdda907e958091e2606: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:55.590740 14826 tablet_bootstrap.cc:492] T 8db6253f6f624f36a4a76af715b9bb73 P d8af8d74ff8b451e80081f46f94504a2: No bootstrap required, opened a new log
I20260811 11:55:55.590889 14826 ts_tablet_manager.cc:1397] T 8db6253f6f624f36a4a76af715b9bb73 P d8af8d74ff8b451e80081f46f94504a2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 11:55:55.591315 14827 tablet_bootstrap.cc:492] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe: No bootstrap required, opened a new log
I20260811 11:55:55.591398 14827 ts_tablet_manager.cc:1397] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:55.591418 14826 raft_consensus.cc:348] T 8db6253f6f624f36a4a76af715b9bb73 P d8af8d74ff8b451e80081f46f94504a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8af8d74ff8b451e80081f46f94504a2" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42641 } } peers { permanent_uuid: "b063e255083e4cdda907e958091e2606" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39527 } } peers { permanent_uuid: "0e664a94aff147c5aea4d0c580f610fe" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42017 } }
I20260811 11:55:55.591542 14826 raft_consensus.cc:374] T 8db6253f6f624f36a4a76af715b9bb73 P d8af8d74ff8b451e80081f46f94504a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:55.591583 14826 raft_consensus.cc:729] T 8db6253f6f624f36a4a76af715b9bb73 P d8af8d74ff8b451e80081f46f94504a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8af8d74ff8b451e80081f46f94504a2, State: Initialized, Role: FOLLOWER
I20260811 11:55:55.591711 14826 consensus_queue.cc:260] T 8db6253f6f624f36a4a76af715b9bb73 P d8af8d74ff8b451e80081f46f94504a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8af8d74ff8b451e80081f46f94504a2" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42641 } } peers { permanent_uuid: "b063e255083e4cdda907e958091e2606" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39527 } } peers { permanent_uuid: "0e664a94aff147c5aea4d0c580f610fe" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42017 } }
I20260811 11:55:55.591945 14827 raft_consensus.cc:348] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8af8d74ff8b451e80081f46f94504a2" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42641 } } peers { permanent_uuid: "b063e255083e4cdda907e958091e2606" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39527 } } peers { permanent_uuid: "0e664a94aff147c5aea4d0c580f610fe" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42017 } }
I20260811 11:55:55.592064 14827 raft_consensus.cc:374] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:55.592098 14827 raft_consensus.cc:729] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e664a94aff147c5aea4d0c580f610fe, State: Initialized, Role: FOLLOWER
I20260811 11:55:55.592298 14828 tablet_bootstrap.cc:492] T 8db6253f6f624f36a4a76af715b9bb73 P b063e255083e4cdda907e958091e2606: No bootstrap required, opened a new log
I20260811 11:55:55.592242 14827 consensus_queue.cc:260] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8af8d74ff8b451e80081f46f94504a2" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42641 } } peers { permanent_uuid: "b063e255083e4cdda907e958091e2606" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39527 } } peers { permanent_uuid: "0e664a94aff147c5aea4d0c580f610fe" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42017 } }
I20260811 11:55:55.592370 14828 ts_tablet_manager.cc:1397] T 8db6253f6f624f36a4a76af715b9bb73 P b063e255083e4cdda907e958091e2606: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 11:55:55.592597 14827 ts_tablet_manager.cc:1428] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 11:55:55.592865 14828 raft_consensus.cc:348] T 8db6253f6f624f36a4a76af715b9bb73 P b063e255083e4cdda907e958091e2606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8af8d74ff8b451e80081f46f94504a2" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42641 } } peers { permanent_uuid: "b063e255083e4cdda907e958091e2606" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39527 } } peers { permanent_uuid: "0e664a94aff147c5aea4d0c580f610fe" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42017 } }
I20260811 11:55:55.592985 14828 raft_consensus.cc:374] T 8db6253f6f624f36a4a76af715b9bb73 P b063e255083e4cdda907e958091e2606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:55.593022 14828 raft_consensus.cc:729] T 8db6253f6f624f36a4a76af715b9bb73 P b063e255083e4cdda907e958091e2606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b063e255083e4cdda907e958091e2606, State: Initialized, Role: FOLLOWER
I20260811 11:55:55.593149 14828 consensus_queue.cc:260] T 8db6253f6f624f36a4a76af715b9bb73 P b063e255083e4cdda907e958091e2606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8af8d74ff8b451e80081f46f94504a2" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42641 } } peers { permanent_uuid: "b063e255083e4cdda907e958091e2606" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39527 } } peers { permanent_uuid: "0e664a94aff147c5aea4d0c580f610fe" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42017 } }
I20260811 11:55:55.593482 14828 ts_tablet_manager.cc:1428] T 8db6253f6f624f36a4a76af715b9bb73 P b063e255083e4cdda907e958091e2606: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:55:55.593645 14826 ts_tablet_manager.cc:1428] T 8db6253f6f624f36a4a76af715b9bb73 P d8af8d74ff8b451e80081f46f94504a2: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260811 11:55:55.595197 14726 heartbeater.cc:503] Master 127.6.143.189:43751 was elected leader, sending a full tablet report...
I20260811 11:55:55.595970 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.593950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.6.143.129:41675)
0811 11:55:55.595206 (+ 1256us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:55.595272 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1186,"thread_start_us":88,"threads_started":1}
I20260811 11:55:55.596215 14834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.592309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.6.143.131:33817)
0811 11:55:55.596041 (+ 3732us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:55.596112 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3664,"thread_start_us":123,"threads_started":1}
I20260811 11:55:55.596509 14836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.592972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.6.143.130:47051)
0811 11:55:55.596338 (+ 3366us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:55.596420 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3296,"thread_start_us":90,"threads_started":1}
I20260811 11:55:55.597119 14834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.596862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.6.143.131:60883)
0811 11:55:55.596983 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:55.597028 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 11:55:55.597150 14579 heartbeater.cc:503] Master 127.6.143.189:43751 was elected leader, sending a full tablet report...
I20260811 11:55:55.597259 14836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.597051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.6.143.130:39623)
0811 11:55:55.597135 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:55.597190 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260811 11:55:55.598137 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.597849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.6.143.129:43203)
0811 11:55:55.598003 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:55.598078 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260811 11:55:55.598462 14425 heartbeater.cc:503] Master 127.6.143.189:43751 was elected leader, sending a full tablet report...
I20260811 11:55:55.909543 14835 raft_consensus.cc:482] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:55:55.909720 14835 raft_consensus.cc:504] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8af8d74ff8b451e80081f46f94504a2" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42641 } } peers { permanent_uuid: "b063e255083e4cdda907e958091e2606" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39527 } } peers { permanent_uuid: "0e664a94aff147c5aea4d0c580f610fe" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42017 } }
I20260811 11:55:55.910167 14835 leader_election.cc:288] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8af8d74ff8b451e80081f46f94504a2 (127.6.143.131:42641), b063e255083e4cdda907e958091e2606 (127.6.143.129:39527)
I20260811 11:55:55.910701 14672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8db6253f6f624f36a4a76af715b9bb73" candidate_uuid: "0e664a94aff147c5aea4d0c580f610fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8af8d74ff8b451e80081f46f94504a2" is_pre_election: true
I20260811 11:55:55.910701 14374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8db6253f6f624f36a4a76af715b9bb73" candidate_uuid: "0e664a94aff147c5aea4d0c580f610fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b063e255083e4cdda907e958091e2606" is_pre_election: true
I20260811 11:55:55.910866 14672 raft_consensus.cc:2454] T 8db6253f6f624f36a4a76af715b9bb73 P d8af8d74ff8b451e80081f46f94504a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e664a94aff147c5aea4d0c580f610fe in term 0.
I20260811 11:55:55.910866 14374 raft_consensus.cc:2454] T 8db6253f6f624f36a4a76af715b9bb73 P b063e255083e4cdda907e958091e2606 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0e664a94aff147c5aea4d0c580f610fe in term 0.
I20260811 11:55:55.911235 14456 leader_election.cc:302] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0e664a94aff147c5aea4d0c580f610fe, b063e255083e4cdda907e958091e2606, d8af8d74ff8b451e80081f46f94504a2; no voters:
I20260811 11:55:55.911420 14835 raft_consensus.cc:2781] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:55:55.911494 14835 raft_consensus.cc:482] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:55:55.911535 14835 raft_consensus.cc:3037] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:55.912678 14835 raft_consensus.cc:504] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8af8d74ff8b451e80081f46f94504a2" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42641 } } peers { permanent_uuid: "b063e255083e4cdda907e958091e2606" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39527 } } peers { permanent_uuid: "0e664a94aff147c5aea4d0c580f610fe" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42017 } }
I20260811 11:55:55.913012 14835 leader_election.cc:288] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe [CANDIDATE]: Term 1 election: Requested vote from peers d8af8d74ff8b451e80081f46f94504a2 (127.6.143.131:42641), b063e255083e4cdda907e958091e2606 (127.6.143.129:39527)
I20260811 11:55:55.913398 14374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8db6253f6f624f36a4a76af715b9bb73" candidate_uuid: "0e664a94aff147c5aea4d0c580f610fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b063e255083e4cdda907e958091e2606"
I20260811 11:55:55.913394 14672 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8db6253f6f624f36a4a76af715b9bb73" candidate_uuid: "0e664a94aff147c5aea4d0c580f610fe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8af8d74ff8b451e80081f46f94504a2"
I20260811 11:55:55.913537 14374 raft_consensus.cc:3037] T 8db6253f6f624f36a4a76af715b9bb73 P b063e255083e4cdda907e958091e2606 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:55.913551 14672 raft_consensus.cc:3037] T 8db6253f6f624f36a4a76af715b9bb73 P d8af8d74ff8b451e80081f46f94504a2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:55.914597 14672 raft_consensus.cc:2454] T 8db6253f6f624f36a4a76af715b9bb73 P d8af8d74ff8b451e80081f46f94504a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e664a94aff147c5aea4d0c580f610fe in term 1.
I20260811 11:55:55.914726 14374 raft_consensus.cc:2454] T 8db6253f6f624f36a4a76af715b9bb73 P b063e255083e4cdda907e958091e2606 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0e664a94aff147c5aea4d0c580f610fe in term 1.
I20260811 11:55:55.914920 14453 leader_election.cc:302] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0e664a94aff147c5aea4d0c580f610fe, d8af8d74ff8b451e80081f46f94504a2; no voters:
I20260811 11:55:55.915091 14835 raft_consensus.cc:2781] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:55.915328 14835 raft_consensus.cc:686] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe [term 1 LEADER]: Becoming Leader. State: Replica: 0e664a94aff147c5aea4d0c580f610fe, State: Running, Role: LEADER
I20260811 11:55:55.915453 14835 consensus_queue.cc:237] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8af8d74ff8b451e80081f46f94504a2" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42641 } } peers { permanent_uuid: "b063e255083e4cdda907e958091e2606" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39527 } } peers { permanent_uuid: "0e664a94aff147c5aea4d0c580f610fe" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42017 } }
I20260811 11:55:55.916432 14836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.915829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.6.143.130:43771)
0811 11:55:55.916216 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:55.916279 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303}
I20260811 11:55:55.917325 14021 catalog_manager.cc:5495] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe reported cstate change: term changed from 0 to 1, leader changed from <none> to 0e664a94aff147c5aea4d0c580f610fe (127.6.143.130). New cstate: current_term: 1 leader_uuid: "0e664a94aff147c5aea4d0c580f610fe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8af8d74ff8b451e80081f46f94504a2" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b063e255083e4cdda907e958091e2606" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0e664a94aff147c5aea4d0c580f610fe" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42017 } health_report { overall_health: HEALTHY } } }
I20260811 11:55:55.944906 14825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.942086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42506 (local address 127.6.143.130:42017)
0811 11:55:55.942215 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:55.942219 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:55.942232 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:55:55.942360 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:55.942363 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:55.942407 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:55.942473 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:55:55.942691 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:55.943182 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:55.944030 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:55.944237 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:55.944316 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:55.944536 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:55:55.944604 (+ 68us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:55:55.944657 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:55.944756 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 11:55:55.944801 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 11:55:55.946928 14672 raft_consensus.cc:1261] T 8db6253f6f624f36a4a76af715b9bb73 P d8af8d74ff8b451e80081f46f94504a2 [term 1 FOLLOWER]: Refusing update from remote peer 0e664a94aff147c5aea4d0c580f610fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:55.946945 14374 raft_consensus.cc:1261] T 8db6253f6f624f36a4a76af715b9bb73 P b063e255083e4cdda907e958091e2606 [term 1 FOLLOWER]: Refusing update from remote peer 0e664a94aff147c5aea4d0c580f610fe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:55.947402 14839 consensus_queue.cc:1035] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "b063e255083e4cdda907e958091e2606" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:55.947557 14835 consensus_queue.cc:1035] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8af8d74ff8b451e80081f46f94504a2" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:55.949328 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.949031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.6.143.129:45915)
0811 11:55:55.949198 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:55.949255 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260811 11:55:55.950119 14834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.949882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.6.143.131:47253)
0811 11:55:55.949984 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:55.950042 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 11:55:55.952901 14838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.952551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.6.143.129:37675)
0811 11:55:55.952763 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:55.952818 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
I20260811 11:55:55.953536 14834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.952833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.6.143.131:50803)
0811 11:55:55.953422 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:55.953475 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":519}
I20260811 11:55:55.954516 14836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.952706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46619 (local address 127.6.143.130:53831)
0811 11:55:55.954367 (+ 1661us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:55.954435 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46619: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1584}
I20260811 11:55:55.958452 14021 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:55:55.958609 14021 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:55:55.962023 14021 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 79d3c932590e456c86fee8a8faecbec3: Sending DeleteTablet for 3 replicas of tablet 8db6253f6f624f36a4a76af715b9bb73
I20260811 11:55:55.962675 14652 tablet_service.cc:1514] Processing DeleteTablet for tablet 8db6253f6f624f36a4a76af715b9bb73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:55 UTC) from {username='oryx', principal='oryx/127.6.143.189@KRBTEST.COM'} at 127.0.0.1:43642
I20260811 11:55:55.962710 14353 tablet_service.cc:1514] Processing DeleteTablet for tablet 8db6253f6f624f36a4a76af715b9bb73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:55 UTC) from {username='oryx', principal='oryx/127.6.143.189@KRBTEST.COM'} at 127.0.0.1:32908
I20260811 11:55:55.962877 14502 tablet_service.cc:1514] Processing DeleteTablet for tablet 8db6253f6f624f36a4a76af715b9bb73 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:55 UTC) from {username='oryx', principal='oryx/127.6.143.189@KRBTEST.COM'} at 127.0.0.1:42496
I20260811 11:55:55.963085 14856 tablet_replica.cc:330] T 8db6253f6f624f36a4a76af715b9bb73 P b063e255083e4cdda907e958091e2606: stopping tablet replica
I20260811 11:55:55.963218 14856 raft_consensus.cc:2229] T 8db6253f6f624f36a4a76af715b9bb73 P b063e255083e4cdda907e958091e2606 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:55:55.963235 14857 tablet_replica.cc:330] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe: stopping tablet replica
I20260811 11:55:55.963352 14856 raft_consensus.cc:2258] T 8db6253f6f624f36a4a76af715b9bb73 P b063e255083e4cdda907e958091e2606 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:55.963393 14857 raft_consensus.cc:2229] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:55:55.963587 14857 raft_consensus.cc:2258] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:55.963802 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 14186
I20260811 11:55:55.965129 14855 tablet_replica.cc:330] T 8db6253f6f624f36a4a76af715b9bb73 P d8af8d74ff8b451e80081f46f94504a2: stopping tablet replica
I20260811 11:55:55.965219 14855 raft_consensus.cc:2229] T 8db6253f6f624f36a4a76af715b9bb73 P d8af8d74ff8b451e80081f46f94504a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:55:55.965304 14855 raft_consensus.cc:2258] T 8db6253f6f624f36a4a76af715b9bb73 P d8af8d74ff8b451e80081f46f94504a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:55.965978 14855 ts_tablet_manager.cc:1905] T 8db6253f6f624f36a4a76af715b9bb73 P d8af8d74ff8b451e80081f46f94504a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:55.968005 14855 ts_tablet_manager.cc:1918] T 8db6253f6f624f36a4a76af715b9bb73 P d8af8d74ff8b451e80081f46f94504a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:55:55.968075 14855 log.cc:1196] T 8db6253f6f624f36a4a76af715b9bb73 P d8af8d74ff8b451e80081f46f94504a2: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-2/wal/wals/8db6253f6f624f36a4a76af715b9bb73
I20260811 11:55:55.968052 14857 ts_tablet_manager.cc:1905] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:55.968297 14855 ts_tablet_manager.cc:1939] T 8db6253f6f624f36a4a76af715b9bb73 P d8af8d74ff8b451e80081f46f94504a2: Deleting consensus metadata
I20260811 11:55:55.968878 14004 catalog_manager.cc:4841] TS d8af8d74ff8b451e80081f46f94504a2 (127.6.143.131:42641): tablet 8db6253f6f624f36a4a76af715b9bb73 (table test-table [id=474b20dce8794e9fb649efe4917837fc]) successfully deleted
I20260811 11:55:55.971262 14857 ts_tablet_manager.cc:1918] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:55:55.971369 14857 log.cc:1196] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786449319379472-6718-0/minicluster-data/ts-1/wal/wals/8db6253f6f624f36a4a76af715b9bb73
I20260811 11:55:55.971689 14857 ts_tablet_manager.cc:1939] T 8db6253f6f624f36a4a76af715b9bb73 P 0e664a94aff147c5aea4d0c580f610fe: Deleting consensus metadata
I20260811 11:55:55.972412 14007 catalog_manager.cc:4841] TS 0e664a94aff147c5aea4d0c580f610fe (127.6.143.130:42017): tablet 8db6253f6f624f36a4a76af715b9bb73 (table test-table [id=474b20dce8794e9fb649efe4917837fc]) successfully deleted
I20260811 11:55:55.974362 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 14434
I20260811 11:55:55.975703 14804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:55.974471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:39527 (local address 127.0.0.1:32922)
0811 11:55:55.975539 (+ 1068us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:55.975605 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.129:39527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":961}
W20260811 11:55:55.976152 14007 catalog_manager.cc:4584] TS b063e255083e4cdda907e958091e2606 (127.6.143.129:39527): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8db6253f6f624f36a4a76af715b9bb73: Network error: Client connection negotiation failed: client connection to 127.6.143.129:39527: connect: Connection refused (error 111)
I20260811 11:55:55.983171 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 14585
I20260811 11:55:55.990487 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 13871
I20260811 11:55:55.997954 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 13990
2026-08-11T11:55:56Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (5810 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:55:56 dist-test-slave-zkpd krb5kdc[14860](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:55:56 dist-test-slave-zkpd krb5kdc[14860](info): set up 2 sockets
Aug 11 11:55:56 dist-test-slave-zkpd krb5kdc[14860](info): commencing operation
krb5kdc: starting...
W20260811 11:55:57.078301 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.055s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:55:57 dist-test-slave-zkpd krb5kdc[14860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:55:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:55:57Z Disabled control of system clock
WARNING: no policy specified for oryx/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.190.keytab.
Entry for principal oryx/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.190.keytab.
I20260811 11:55:57.200189 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:39159
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.190.keytab
--principal=oryx/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:44075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:39159
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:57.314271 14876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:57.314527 14876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:57.314567 14876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:57.318590 14876 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:55:57.318642 14876 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 11:55:57.318660 14876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:57.318677 14876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:57.318693 14876 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:55:57.318708 14876 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:55:57.323923 14876 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:44075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:39159
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.190.keytab
--principal=oryx/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:39159
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:57.324898 14876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:57.325878 14876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:57.334342 14882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:57.334441 14885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:57.334429 14876 server_base.cc:1034] running on GCE node
W20260811 11:55:57.334352 14883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:57.335263 14876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:57.336390 14876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:57.337561 14876 hybrid_clock.cc:648] HybridClock initialized: now 1786449357337535 us; error 38 us; skew 500 ppm
Aug 11 11:55:57 dist-test-slave-zkpd krb5kdc[14860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449357, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:57.344491 14876 init.cc:377] Logged in from keytab as oryx/127.6.143.190@KRBTEST.COM (short username oryx)
I20260811 11:55:57.345486 14876 webserver.cc:460] Webserver started at http://127.6.143.190:40789/ using document root <none> and password file <none>
I20260811 11:55:57.346065 14876 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:57.346128 14876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:57.346372 14876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:57.348510 14876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "f9ce963fd5964673929ebfb3cfbf4f27"
format_stamp: "Formatted at 2026-08-11 11:55:57 on dist-test-slave-zkpd"
server_key: "e412d640892b2be05a8731c2c19ed57f"
server_key_iv: "c2f0b46bc0a98ea00df62a733e7f18d1"
server_key_version: "encryptionkey@0"
I20260811 11:55:57.349084 14876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "f9ce963fd5964673929ebfb3cfbf4f27"
format_stamp: "Formatted at 2026-08-11 11:55:57 on dist-test-slave-zkpd"
server_key: "e412d640892b2be05a8731c2c19ed57f"
server_key_iv: "c2f0b46bc0a98ea00df62a733e7f18d1"
server_key_version: "encryptionkey@0"
I20260811 11:55:57.352856 14876 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 11:55:57.355146 14892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:57.356438 14876 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 11:55:57.356581 14876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "f9ce963fd5964673929ebfb3cfbf4f27"
format_stamp: "Formatted at 2026-08-11 11:55:57 on dist-test-slave-zkpd"
server_key: "e412d640892b2be05a8731c2c19ed57f"
server_key_iv: "c2f0b46bc0a98ea00df62a733e7f18d1"
server_key_version: "encryptionkey@0"
I20260811 11:55:57.356690 14876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:57.402612 14876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:57.405692 14876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:57.405905 14876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:57.414103 14876 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:39159
I20260811 11:55:57.414134 14962 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:39159 every 8 connection(s)
I20260811 11:55:57.415050 14876 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:55:57.416267 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 14876
I20260811 11:55:57.416378 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:55:57.416677 6718 external_mini_cluster.cc:1428] Setting key ce38fc6aa30101ca70ad1be8ebb4ff55
I20260811 11:55:57.418246 14963 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:57.423923 14963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9ce963fd5964673929ebfb3cfbf4f27: Bootstrap starting.
Aug 11 11:55:57 dist-test-slave-zkpd krb5kdc[14860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449357, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM
I20260811 11:55:57.426363 14963 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9ce963fd5964673929ebfb3cfbf4f27: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:57.427608 14963 log.cc:824] T 00000000000000000000000000000000 P f9ce963fd5964673929ebfb3cfbf4f27: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:57.429816 14963 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9ce963fd5964673929ebfb3cfbf4f27: No bootstrap required, opened a new log
I20260811 11:55:57.430968 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:57.418062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55180 (local address 127.6.143.190:39159)
0811 11:55:57.418524 (+ 462us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:57.418539 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:57.418574 (+ 35us) server_negotiation.cc:407] Connection header received
0811 11:55:57.419208 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:57.419224 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:57.419551 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:57.419887 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:57.420718 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:57.421706 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:57.422706 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:57.423231 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:57.423410 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:57.426192 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:57.426198 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:57.426212 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:57.426238 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:57.428153 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:57.428694 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:57.428699 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:57.428703 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:57.428811 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:57.429107 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:57.429109 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:57.429110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:57.429460 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:57.429596 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:57.429945 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 11:55:57.430284 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":138,"threads_started":1}
I20260811 11:55:57.432933 14963 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9ce963fd5964673929ebfb3cfbf4f27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9ce963fd5964673929ebfb3cfbf4f27" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39159 } }
I20260811 11:55:57.433137 14963 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9ce963fd5964673929ebfb3cfbf4f27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:57.433220 14963 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9ce963fd5964673929ebfb3cfbf4f27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9ce963fd5964673929ebfb3cfbf4f27, State: Initialized, Role: FOLLOWER
I20260811 11:55:57.433662 14963 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9ce963fd5964673929ebfb3cfbf4f27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9ce963fd5964673929ebfb3cfbf4f27" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39159 } }
I20260811 11:55:57.433833 14963 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9ce963fd5964673929ebfb3cfbf4f27 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:55:57.433904 14963 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9ce963fd5964673929ebfb3cfbf4f27 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:55:57.434019 14963 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9ce963fd5964673929ebfb3cfbf4f27 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:57.434850 14963 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9ce963fd5964673929ebfb3cfbf4f27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9ce963fd5964673929ebfb3cfbf4f27" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39159 } }
I20260811 11:55:57.435195 14963 leader_election.cc:302] T 00000000000000000000000000000000 P f9ce963fd5964673929ebfb3cfbf4f27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9ce963fd5964673929ebfb3cfbf4f27; no voters:
I20260811 11:55:57.435516 14963 leader_election.cc:288] T 00000000000000000000000000000000 P f9ce963fd5964673929ebfb3cfbf4f27 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:55:57.435627 14968 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9ce963fd5964673929ebfb3cfbf4f27 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:57.435848 14968 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9ce963fd5964673929ebfb3cfbf4f27 [term 1 LEADER]: Becoming Leader. State: Replica: f9ce963fd5964673929ebfb3cfbf4f27, State: Running, Role: LEADER
I20260811 11:55:57.436223 14968 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9ce963fd5964673929ebfb3cfbf4f27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9ce963fd5964673929ebfb3cfbf4f27" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39159 } }
I20260811 11:55:57.436718 14963 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9ce963fd5964673929ebfb3cfbf4f27 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:55:57.437860 14971 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9ce963fd5964673929ebfb3cfbf4f27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9ce963fd5964673929ebfb3cfbf4f27. Latest consensus state: current_term: 1 leader_uuid: "f9ce963fd5964673929ebfb3cfbf4f27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9ce963fd5964673929ebfb3cfbf4f27" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39159 } } }
I20260811 11:55:57.437988 14971 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9ce963fd5964673929ebfb3cfbf4f27 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:57.438287 14970 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9ce963fd5964673929ebfb3cfbf4f27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9ce963fd5964673929ebfb3cfbf4f27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9ce963fd5964673929ebfb3cfbf4f27" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39159 } } }
I20260811 11:55:57.438385 14978 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:55:57.438385 14970 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9ce963fd5964673929ebfb3cfbf4f27 [sys.catalog]: This master's current role is: LEADER
I20260811 11:55:57.441538 14978 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:55:57.447310 14978 catalog_manager.cc:1347] Generated new cluster ID: 09bfcc6b2ab44a88ad1490aa24f5c3f5
I20260811 11:55:57.447382 14978 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:55:57.466601 14978 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:55:57.467773 14978 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:55:57.481745 14978 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9ce963fd5964673929ebfb3cfbf4f27: Generated new TSK 0
I20260811 11:55:57.482604 14978 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.6.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.129.keytab.
Entry for principal oryx/127.6.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.129.keytab.
I20260811 11:55:57.511046 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:0
--local_ip_for_outbound_sockets=127.6.143.129
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:39159
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.129.keytab
--principal=oryx/127.6.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:44075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:57.623462 14991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:57.623694 14991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:57.623757 14991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:57.627714 14991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:57.627767 14991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:57.627806 14991 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 11:55:57.627837 14991 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:55:57.632836 14991 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:44075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.129.keytab
--principal=oryx/127.6.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.143.190:39159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:57.633845 14991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:57.634828 14991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:57.644848 14996 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:57.644812 14998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:57.644907 14991 server_base.cc:1034] running on GCE node
W20260811 11:55:57.645066 15000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:57.645502 14991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:57.646168 14991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:57.647329 14991 hybrid_clock.cc:648] HybridClock initialized: now 1786449357647299 us; error 41 us; skew 500 ppm
Aug 11 11:55:57 dist-test-slave-zkpd krb5kdc[14860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449357, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:57.650128 14991 init.cc:377] Logged in from keytab as oryx/127.6.143.129@KRBTEST.COM (short username oryx)
I20260811 11:55:57.650981 14991 webserver.cc:460] Webserver started at http://127.6.143.129:36365/ using document root <none> and password file <none>
I20260811 11:55:57.651551 14991 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:57.651614 14991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:57.651854 14991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:57.653921 14991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-0/data/instance:
uuid: "f0a368c2bba649cfa232d7321568e655"
format_stamp: "Formatted at 2026-08-11 11:55:57 on dist-test-slave-zkpd"
server_key: "7c4daa9bca54a0275057158d26214ea1"
server_key_iv: "f5730f5ecc5de57a0c7a33bcd39df68c"
server_key_version: "encryptionkey@0"
I20260811 11:55:57.654495 14991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance:
uuid: "f0a368c2bba649cfa232d7321568e655"
format_stamp: "Formatted at 2026-08-11 11:55:57 on dist-test-slave-zkpd"
server_key: "7c4daa9bca54a0275057158d26214ea1"
server_key_iv: "f5730f5ecc5de57a0c7a33bcd39df68c"
server_key_version: "encryptionkey@0"
I20260811 11:55:57.658376 14991 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 11:55:57.661023 15008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:57.662271 14991 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 11:55:57.662408 14991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "f0a368c2bba649cfa232d7321568e655"
format_stamp: "Formatted at 2026-08-11 11:55:57 on dist-test-slave-zkpd"
server_key: "7c4daa9bca54a0275057158d26214ea1"
server_key_iv: "f5730f5ecc5de57a0c7a33bcd39df68c"
server_key_version: "encryptionkey@0"
I20260811 11:55:57.662518 14991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:57.693634 14991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:57.696419 14991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:57.696615 14991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:57.698007 14991 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:57.698081 14991 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:57.698153 14991 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:57.698206 14991 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 11:55:57 dist-test-slave-zkpd krb5kdc[14860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449357, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM
I20260811 11:55:57.709331 14991 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:40905
I20260811 11:55:57.709435 15137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:40905 every 8 connection(s)
I20260811 11:55:57.710498 14991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:55:57.712512 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:57.699009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:42933 (local address 127.6.143.190:39159)
0811 11:55:57.699173 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:57.699178 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:57.699969 (+ 791us) server_negotiation.cc:407] Connection header received
0811 11:55:57.700990 (+ 1021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:57.700997 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:57.701062 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:57.701122 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:57.702511 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:57.703174 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:57.704614 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:57.704989 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:57.705119 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:57.708676 (+ 3557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:57.708684 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:57.708690 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:57.708722 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:57.710432 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:57.710984 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:57.710989 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:57.710991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:57.711088 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:57.711500 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:57.711502 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:57.711504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:57.711710 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:57.711790 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:57.712285 (+ 495us) server_negotiation.cc:299] Negotiation successful
0811 11:55:57.712388 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 11:55:57.713248 15017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:57.699360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39159 (local address 127.6.143.129:42933)
0811 11:55:57.699792 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:57.699842 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:57.700720 (+ 878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:57.701604 (+ 884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:57.701610 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:57.701915 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:57.702363 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:57.702377 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:57.703304 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:57.703309 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:57.704482 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:57.704493 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:57.705170 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:57.705175 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:57.705362 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:57.705909 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:57.705936 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:57.708500 (+ 2564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:57.710570 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:57.710577 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:57.710595 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:57.710866 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:57.711198 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:57.711203 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:57.711205 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:57.711377 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:57.711804 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:57.711809 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:57.712070 (+ 261us) client_negotiation.cc:769] Sending connection context
0811 11:55:57.712214 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 11:55:57.712455 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":123,"threads_started":1}
I20260811 11:55:57.714479 15138 heartbeater.cc:347] Connected to a master server at 127.6.143.190:39159
I20260811 11:55:57.714617 15138 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:57.715144 15138 heartbeater.cc:511] Master 127.6.143.190:39159 requested a full tablet report, sending...
I20260811 11:55:57.716773 14910 ts_manager.cc:194] Registered new tserver with Master: f0a368c2bba649cfa232d7321568e655 (127.6.143.129:40905)
I20260811 11:55:57.717708 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 14991
I20260811 11:55:57.717877 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance
I20260811 11:55:57.718205 6718 external_mini_cluster.cc:1428] Setting key 566780b1e07e8a0d7a7d3fa70c0b648b
I20260811 11:55:57.718230 14910 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.143.129@KRBTEST.COM'} at 127.6.143.129:42933
I20260811 11:55:57.724808 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:57.716815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:38155 (local address 127.6.143.190:39159)
0811 11:55:57.716928 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:57.716931 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:57.717143 (+ 212us) server_negotiation.cc:407] Connection header received
0811 11:55:57.717245 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:57.717247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:57.717296 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:57.717381 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:57.717911 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:57.718607 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:57.719815 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:57.720154 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:57.720262 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:57.721546 (+ 1284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:57.721549 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:57.721551 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:57.721572 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:57.723170 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:57.723599 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:57.723604 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:57.723605 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:57.723687 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:57.724007 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:57.724011 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:57.724013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:57.724182 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:57.724289 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:57.724552 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 11:55:57.724659 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 11:55:57.724822 15144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:57.716732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39159 (local address 127.6.143.129:38155)
0811 11:55:57.717063 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:57.717078 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:57.717167 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:57.717410 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:57.717413 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:57.717625 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:57.717737 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:57.717743 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:57.718925 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:57.718928 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:57.719629 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:57.719658 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:57.720284 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:57.720290 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:57.720409 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:57.720929 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:57.720949 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:57.721408 (+ 459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:57.723278 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:57.723282 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:57.723284 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:57.723498 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:57.723777 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:57.723780 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:57.723782 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:57.723867 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:57.724280 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:57.724283 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:57.724382 (+ 99us) client_negotiation.cc:769] Sending connection context
0811 11:55:57.724497 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 11:55:57.724631 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":1,"thread_start_us":92,"threads_started":1}
WARNING: no policy specified for oryx/127.6.143.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.143.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.130.keytab.
Entry for principal oryx/127.6.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.130.keytab.
I20260811 11:55:57.744678 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:0
--local_ip_for_outbound_sockets=127.6.143.130
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:39159
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.130.keytab
--principal=oryx/127.6.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:44075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:57.856984 15147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:57.857209 15147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:57.857251 15147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:57.861248 15147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:57.861296 15147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:57.861361 15147 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 11:55:57.861429 15147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:55:57.866430 15147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:44075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.130.keytab
--principal=oryx/127.6.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.143.190:39159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:57.867681 15147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:57.869000 15147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:57.878492 15156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:57.878592 15147 server_base.cc:1034] running on GCE node
W20260811 11:55:57.878696 15154 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:57.878482 15153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:57.879124 15147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:57.879741 15147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:57.880896 15147 hybrid_clock.cc:648] HybridClock initialized: now 1786449357880877 us; error 33 us; skew 500 ppm
Aug 11 11:55:57 dist-test-slave-zkpd krb5kdc[14860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449357, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:57.883677 15147 init.cc:377] Logged in from keytab as oryx/127.6.143.130@KRBTEST.COM (short username oryx)
I20260811 11:55:57.884552 15147 webserver.cc:460] Webserver started at http://127.6.143.130:35965/ using document root <none> and password file <none>
I20260811 11:55:57.885092 15147 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:57.885152 15147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:57.885385 15147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:57.887336 15147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-1/data/instance:
uuid: "87106c511d364f078ea5c1aa4a8be5dd"
format_stamp: "Formatted at 2026-08-11 11:55:57 on dist-test-slave-zkpd"
server_key: "a4600b5fe13db9e1f8acec567a4e1dfa"
server_key_iv: "f0c11038daccf2be0b401c0cd157c68b"
server_key_version: "encryptionkey@0"
I20260811 11:55:57.887929 15147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance:
uuid: "87106c511d364f078ea5c1aa4a8be5dd"
format_stamp: "Formatted at 2026-08-11 11:55:57 on dist-test-slave-zkpd"
server_key: "a4600b5fe13db9e1f8acec567a4e1dfa"
server_key_iv: "f0c11038daccf2be0b401c0cd157c68b"
server_key_version: "encryptionkey@0"
I20260811 11:55:57.891412 15147 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 11:55:57.893826 15165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:57.894977 15147 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:57.895097 15147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "87106c511d364f078ea5c1aa4a8be5dd"
format_stamp: "Formatted at 2026-08-11 11:55:57 on dist-test-slave-zkpd"
server_key: "a4600b5fe13db9e1f8acec567a4e1dfa"
server_key_iv: "f0c11038daccf2be0b401c0cd157c68b"
server_key_version: "encryptionkey@0"
I20260811 11:55:57.895190 15147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:57.916201 15147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:57.919275 15147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:57.919477 15147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:57.920828 15147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:57.920877 15147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:57.920915 15147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:57.920970 15147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 11:55:57 dist-test-slave-zkpd krb5kdc[14860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449357, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM
I20260811 11:55:57.932025 15147 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:37463
I20260811 11:55:57.932430 15293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:37463 every 8 connection(s)
I20260811 11:55:57.932958 15147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
I20260811 11:55:57.935209 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:57.921884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:54103 (local address 127.6.143.190:39159)
0811 11:55:57.922043 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:57.922048 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:57.922798 (+ 750us) server_negotiation.cc:407] Connection header received
0811 11:55:57.923987 (+ 1189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:57.923990 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:57.924051 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:57.924144 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:57.925748 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:57.926377 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:57.928216 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:57.928525 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:57.928639 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:57.931669 (+ 3030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:57.931673 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:57.931676 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:57.931703 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:57.933395 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:57.933973 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:57.933976 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:57.933977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:57.934054 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:57.934315 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:57.934318 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:57.934319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:57.934482 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:57.934558 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:57.934996 (+ 438us) server_negotiation.cc:299] Negotiation successful
0811 11:55:57.935091 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 11:55:57.936015 15175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:57.922197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39159 (local address 127.6.143.130:54103)
0811 11:55:57.922646 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:57.922693 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:57.923739 (+ 1046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:57.924359 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:57.924371 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:57.924832 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:57.925556 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:57.925583 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:57.926513 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:57.926517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:57.928088 (+ 1571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:57.928104 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:57.928701 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:57.928708 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:57.928929 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:57.929483 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:57.929506 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:57.931477 (+ 1971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:57.933532 (+ 2055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:57.933537 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:57.933550 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:57.933869 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:57.934123 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:57.934125 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:57.934127 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:57.934239 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:57.934574 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:57.934578 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:57.934781 (+ 203us) client_negotiation.cc:769] Sending connection context
0811 11:55:57.934933 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 11:55:57.935156 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":137,"threads_started":1}
I20260811 11:55:57.937402 15294 heartbeater.cc:347] Connected to a master server at 127.6.143.190:39159
I20260811 11:55:57.937573 15294 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:57.938195 15294 heartbeater.cc:511] Master 127.6.143.190:39159 requested a full tablet report, sending...
I20260811 11:55:57.939316 14910 ts_manager.cc:194] Registered new tserver with Master: 87106c511d364f078ea5c1aa4a8be5dd (127.6.143.130:37463)
I20260811 11:55:57.940086 14910 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.143.130@KRBTEST.COM'} at 127.6.143.130:54103
I20260811 11:55:57.940971 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 15147
I20260811 11:55:57.941087 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance
I20260811 11:55:57.941372 6718 external_mini_cluster.cc:1428] Setting key 8e4a2175cb1793cbd286c67c506437d0
I20260811 11:55:57.948945 15302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:57.939627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39159 (local address 127.6.143.130:60451)
0811 11:55:57.939926 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:57.939940 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:57.940036 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:57.940260 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:57.940263 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:57.940463 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:57.940553 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:57.940559 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:57.943007 (+ 2448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:57.943011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:57.943930 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:57.943946 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:57.944386 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:57.944392 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:57.944523 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:57.945110 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:57.945131 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:57.945432 (+ 301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:57.947339 (+ 1907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:57.947341 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:57.947342 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:57.947527 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:57.947825 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:57.947826 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:57.947828 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:57.947963 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:57.948517 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:57.948520 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:57.948620 (+ 100us) client_negotiation.cc:769] Sending connection context
0811 11:55:57.948697 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 11:55:57.948798 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":102,"threads_started":1}
I20260811 11:55:57.949121 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:57.939694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:60451 (local address 127.6.143.190:39159)
0811 11:55:57.939818 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:57.939822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:57.940011 (+ 189us) server_negotiation.cc:407] Connection header received
0811 11:55:57.940116 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:57.940118 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:57.940154 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:57.940245 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:57.940672 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:57.941410 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:57.944069 (+ 2659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:57.944272 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:57.944349 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:57.945573 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:57.945578 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:57.945584 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:57.945614 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:57.947231 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:57.947626 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:57.947632 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:57.947637 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:57.947731 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:57.948054 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:57.948061 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:57.948065 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:57.948395 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:57.948511 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:57.948812 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 11:55:57.948949 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
WARNING: no policy specified for oryx/127.6.143.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.131.keytab.
Entry for principal oryx/127.6.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.131.keytab.
I20260811 11:55:57.969056 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:0
--local_ip_for_outbound_sockets=127.6.143.131
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:39159
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.131.keytab
--principal=oryx/127.6.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:44075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:55:58.081885 15307 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:55:58.082122 15307 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:55:58.082165 15307 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:55:58.086201 15307 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:55:58.086247 15307 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:55:58.086302 15307 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 11:55:58.086334 15307 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:55:58.091203 15307 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:44075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.131.keytab
--principal=oryx/127.6.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.143.190:39159
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15307
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:55:58.092171 15307 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:55:58.093171 15307 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:55:58.102775 15312 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:58.102789 15315 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:55:58.102972 15313 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:55:58.102826 15307 server_base.cc:1034] running on GCE node
I20260811 11:55:58.103363 15307 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:55:58.104019 15307 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:55:58.105206 15307 hybrid_clock.cc:648] HybridClock initialized: now 1786449358105195 us; error 30 us; skew 500 ppm
Aug 11 11:55:58 dist-test-slave-zkpd krb5kdc[14860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449358, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:55:58.107824 15307 init.cc:377] Logged in from keytab as oryx/127.6.143.131@KRBTEST.COM (short username oryx)
I20260811 11:55:58.108665 15307 webserver.cc:460] Webserver started at http://127.6.143.131:33075/ using document root <none> and password file <none>
I20260811 11:55:58.109163 15307 fs_manager.cc:359] Metadata directory not provided
I20260811 11:55:58.109220 15307 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:55:58.109377 15307 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:55:58.111258 15307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-2/data/instance:
uuid: "0042fe01318b4642937a2a7e5fc7f97f"
format_stamp: "Formatted at 2026-08-11 11:55:58 on dist-test-slave-zkpd"
server_key: "a46faf6626d16a0b9ee2d5702d98d663"
server_key_iv: "b613977040220da3fac4ae48f993f438"
server_key_version: "encryptionkey@0"
I20260811 11:55:58.111749 15307 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance:
uuid: "0042fe01318b4642937a2a7e5fc7f97f"
format_stamp: "Formatted at 2026-08-11 11:55:58 on dist-test-slave-zkpd"
server_key: "a46faf6626d16a0b9ee2d5702d98d663"
server_key_iv: "b613977040220da3fac4ae48f993f438"
server_key_version: "encryptionkey@0"
I20260811 11:55:58.115298 15307 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 11:55:58.117766 15323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:58.118949 15307 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:58.119053 15307 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "0042fe01318b4642937a2a7e5fc7f97f"
format_stamp: "Formatted at 2026-08-11 11:55:58 on dist-test-slave-zkpd"
server_key: "a46faf6626d16a0b9ee2d5702d98d663"
server_key_iv: "b613977040220da3fac4ae48f993f438"
server_key_version: "encryptionkey@0"
I20260811 11:55:58.119135 15307 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:55:58.148195 15307 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:55:58.151129 15307 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:55:58.151288 15307 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:55:58.152750 15307 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:55:58.152801 15307 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:55:58.152874 15307 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:55:58.152906 15307 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 11:55:58 dist-test-slave-zkpd krb5kdc[14860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449358, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM
I20260811 11:55:58.164933 15307 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:46615
I20260811 11:55:58.165261 15457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:46615 every 8 connection(s)
I20260811 11:55:58.166154 15307 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:55:58.168174 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:58.153684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:41183 (local address 127.6.143.190:39159)
0811 11:55:58.153817 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:58.153821 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:58.154921 (+ 1100us) server_negotiation.cc:407] Connection header received
0811 11:55:58.156108 (+ 1187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:58.156112 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:58.156181 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:58.156247 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:58.157737 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.158557 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.159727 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.160149 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.160283 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.164015 (+ 3732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:58.164019 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:58.164022 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:58.164057 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:58.165853 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:58.166453 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:58.166456 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:58.166457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:58.166535 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:58.166895 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:58.166900 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:58.166902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:58.167108 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:58.167226 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:58.167850 (+ 624us) server_negotiation.cc:299] Negotiation successful
0811 11:55:58.168001 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 11:55:58.168926 15336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:58.154266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39159 (local address 127.6.143.131:41183)
0811 11:55:58.154763 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:58.154806 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:58.155809 (+ 1003us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:58.156577 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:58.156583 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:58.156950 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:58.157576 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:58.157597 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.158700 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.158703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:58.159594 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:58.159604 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.160354 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.160362 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:58.160582 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.161301 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:58.161329 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:58.163802 (+ 2473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:58.165964 (+ 2162us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:58.165969 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:58.165981 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:58.166272 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:58.166643 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:58.166646 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:58.166648 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:58.166790 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:58.167393 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:58.167397 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:58.167614 (+ 217us) client_negotiation.cc:769] Sending connection context
0811 11:55:58.167804 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 11:55:58.168148 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":143,"threads_started":1}
I20260811 11:55:58.170161 15458 heartbeater.cc:347] Connected to a master server at 127.6.143.190:39159
I20260811 11:55:58.170331 15458 heartbeater.cc:464] Registering TS with master...
I20260811 11:55:58.170918 15458 heartbeater.cc:511] Master 127.6.143.190:39159 requested a full tablet report, sending...
I20260811 11:55:58.171969 14911 ts_manager.cc:194] Registered new tserver with Master: 0042fe01318b4642937a2a7e5fc7f97f (127.6.143.131:46615)
I20260811 11:55:58.172717 14911 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.143.131@KRBTEST.COM'} at 127.6.143.131:41183
I20260811 11:55:58.175968 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 15307
I20260811 11:55:58.176123 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance
I20260811 11:55:58.176445 6718 external_mini_cluster.cc:1428] Setting key 8e45854c0cfb4021b4c8ff5a07b2fc49
I20260811 11:55:58.178550 6718 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:55:58.179994 15464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:58.172305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39159 (local address 127.6.143.131:48061)
0811 11:55:58.173095 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:58.173117 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:58.173195 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:58.173446 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:58.173449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:58.173644 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:58.173750 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:58.173756 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.174494 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.174497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:58.175044 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:58.175051 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.175490 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.175493 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:58.175582 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.176134 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:58.176145 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:58.176493 (+ 348us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:58.178515 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:58.178518 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:58.178519 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:58.178758 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:58.179052 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:58.179054 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:58.179056 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:58.179134 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:58.179557 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:58.179559 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:58.179631 (+ 72us) client_negotiation.cc:769] Sending connection context
0811 11:55:58.179710 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 11:55:58.179811 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":702,"thread_start_us":100,"threads_started":1}
I20260811 11:55:58.180332 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:58.172394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:48061 (local address 127.6.143.190:39159)
0811 11:55:58.172514 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:58.172517 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:58.173189 (+ 672us) server_negotiation.cc:407] Connection header received
0811 11:55:58.173292 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:58.173297 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:58.173340 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:58.173424 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:58.173866 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.174354 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.175164 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.175375 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.175485 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.176669 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:58.176673 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:58.176675 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:58.176700 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:58.178266 (+ 1566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:58.178861 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:58.178864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:58.178866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:58.178953 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:58.179223 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:58.179227 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:58.179228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:58.179461 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:58.179568 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:58.179795 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 11:55:58.179964 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
Aug 11 11:55:58 dist-test-slave-zkpd krb5kdc[14860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449358, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 11:55:58 dist-test-slave-zkpd krb5kdc[14860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:55:58 dist-test-slave-zkpd krb5kdc[14860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 11:55:58.204043 14966 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:55:58.197917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55188 (local address 127.6.143.190:39159)
0811 11:55:58.198043 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:58.198047 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:58.198058 (+ 11us) server_negotiation.cc:407] Connection header received
0811 11:55:58.198095 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:58.198097 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:58.198131 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:58.198238 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:58.198961 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.199520 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.200436 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.200657 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.200765 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.203923 (+ 3158us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55188: BlockingRecv error: recv got EOF from 127.0.0.1:55188 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 11:55:58 dist-test-slave-zkpd krb5kdc[14860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449358, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM
I20260811 11:55:58.213397 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:58.205452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55196 (local address 127.6.143.190:39159)
0811 11:55:58.205580 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:58.205583 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:58.205682 (+ 99us) server_negotiation.cc:407] Connection header received
0811 11:55:58.205759 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:58.205761 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:58.205796 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:58.205866 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:58.206406 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.206868 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.207577 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.207773 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.207846 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.209979 (+ 2133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:58.209982 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:58.209984 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:58.210006 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:58.211450 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:58.212035 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:58.212040 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:58.212041 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:58.212126 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:58.212419 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:58.212422 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:58.212424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:58.212714 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:58.212863 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:58.213109 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 11:55:58.213222 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 11:55:58.216315 14911 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55196:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:55:58.219328 14911 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:55:58.234001 15481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:58.228034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:40905 (local address 127.0.0.1:39422)
0811 11:55:58.228562 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:58.228611 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:58.228750 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:58.229559 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:58.229565 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:58.229587 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:58.229724 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:58.229730 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.231141 (+ 1411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.231144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:58.232572 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:58.232583 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.233410 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.233413 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:58.233518 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.233554 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 11:55:58.233706 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 11:55:58.233826 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"mutex_wait_us":3,"thread_start_us":73,"threads_started":1}
I20260811 11:55:58.234053 15485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:58.228748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:37463 (local address 127.0.0.1:50462)
0811 11:55:58.229341 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:58.229366 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:58.229461 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:58.230158 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:58.230164 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:58.230178 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:58.230326 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:58.230333 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.231352 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.231355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:58.232554 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:58.232564 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.233706 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.233711 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:58.233793 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.233814 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 11:55:58.233888 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 11:55:58.233945 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"mutex_wait_us":64,"thread_start_us":108,"threads_started":1}
I20260811 11:55:58.234356 15484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:58.228284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39422 (local address 127.6.143.129:40905)
0811 11:55:58.228672 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:58.228681 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:58.228706 (+ 25us) server_negotiation.cc:407] Connection header received
0811 11:55:58.228951 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:58.228959 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:58.229184 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:58.229316 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:58.229851 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.231017 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.232715 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.233285 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.233420 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.233571 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:58.234165 (+ 594us) server_negotiation.cc:299] Negotiation successful
0811 11:55:58.234260 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":87,"threads_started":1}
I20260811 11:55:58.234653 15483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:58.228150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50706 (local address 127.6.143.131:46615)
0811 11:55:58.229124 (+ 974us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:58.229136 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:58.229370 (+ 234us) server_negotiation.cc:407] Connection header received
0811 11:55:58.229592 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:58.229599 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:58.229793 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:58.229922 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:58.230387 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.231422 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.232816 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.233549 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.233653 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.233833 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:58.234388 (+ 555us) server_negotiation.cc:299] Negotiation successful
0811 11:55:58.234509 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":878,"thread_start_us":209,"threads_started":1}
I20260811 11:55:58.235013 15486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:58.228595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:46615 (local address 127.0.0.1:50706)
0811 11:55:58.229284 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:58.229297 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:58.229440 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:58.230057 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:58.230059 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:58.230073 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:58.230209 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:58.230218 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.231534 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.231539 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:58.232703 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:58.232710 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.233913 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.233916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:58.234006 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.234029 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 11:55:58.234801 (+ 772us) client_negotiation.cc:240] Negotiation successful
0811 11:55:58.234889 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"mutex_wait_us":3,"thread_start_us":122,"threads_started":1}
I20260811 11:55:58.235350 15487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:58.228982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50462 (local address 127.6.143.130:37463)
0811 11:55:58.229716 (+ 734us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:58.229725 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:58.229743 (+ 18us) server_negotiation.cc:407] Connection header received
0811 11:55:58.229832 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:58.229836 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:58.230017 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:58.230170 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:58.230449 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.231223 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.232678 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.233514 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.233666 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.233833 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:58.233921 (+ 88us) server_negotiation.cc:299] Negotiation successful
0811 11:55:58.234021 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":102,"threads_started":1}
I20260811 11:55:58.238065 15380 tablet_service.cc:1467] Processing CreateTablet for tablet c33c4c38e22c4e0d9df4a7ffc35168d6 (DEFAULT_TABLE table=test-table [id=4961d3c520cf4fa286ab4aaaf05f0868]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:58.238059 15063 tablet_service.cc:1467] Processing CreateTablet for tablet c33c4c38e22c4e0d9df4a7ffc35168d6 (DEFAULT_TABLE table=test-table [id=4961d3c520cf4fa286ab4aaaf05f0868]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:58.238063 15222 tablet_service.cc:1467] Processing CreateTablet for tablet c33c4c38e22c4e0d9df4a7ffc35168d6 (DEFAULT_TABLE table=test-table [id=4961d3c520cf4fa286ab4aaaf05f0868]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:58.239197 15380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c33c4c38e22c4e0d9df4a7ffc35168d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:58.239197 15222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c33c4c38e22c4e0d9df4a7ffc35168d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:58.239200 15063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c33c4c38e22c4e0d9df4a7ffc35168d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:58.245421 15488 tablet_bootstrap.cc:492] T c33c4c38e22c4e0d9df4a7ffc35168d6 P f0a368c2bba649cfa232d7321568e655: Bootstrap starting.
I20260811 11:55:58.247339 15490 tablet_bootstrap.cc:492] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 0042fe01318b4642937a2a7e5fc7f97f: Bootstrap starting.
I20260811 11:55:58.247816 15488 tablet_bootstrap.cc:654] T c33c4c38e22c4e0d9df4a7ffc35168d6 P f0a368c2bba649cfa232d7321568e655: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:58.248579 15488 log.cc:824] T c33c4c38e22c4e0d9df4a7ffc35168d6 P f0a368c2bba649cfa232d7321568e655: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:58.249148 15490 tablet_bootstrap.cc:654] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 0042fe01318b4642937a2a7e5fc7f97f: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:58.249548 15489 tablet_bootstrap.cc:492] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd: Bootstrap starting.
I20260811 11:55:58.250006 15490 log.cc:824] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 0042fe01318b4642937a2a7e5fc7f97f: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:58.251207 15489 tablet_bootstrap.cc:654] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:58.251767 15490 tablet_bootstrap.cc:492] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 0042fe01318b4642937a2a7e5fc7f97f: No bootstrap required, opened a new log
I20260811 11:55:58.251945 15489 log.cc:824] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd: Log is configured to *not* fsync() on all Append() calls
I20260811 11:55:58.251988 15490 ts_tablet_manager.cc:1397] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 0042fe01318b4642937a2a7e5fc7f97f: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260811 11:55:58.253156 15488 tablet_bootstrap.cc:492] T c33c4c38e22c4e0d9df4a7ffc35168d6 P f0a368c2bba649cfa232d7321568e655: No bootstrap required, opened a new log
I20260811 11:55:58.253345 15488 ts_tablet_manager.cc:1397] T c33c4c38e22c4e0d9df4a7ffc35168d6 P f0a368c2bba649cfa232d7321568e655: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260811 11:55:58.253902 15489 tablet_bootstrap.cc:492] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd: No bootstrap required, opened a new log
I20260811 11:55:58.254081 15489 ts_tablet_manager.cc:1397] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 11:55:58.255307 15490 raft_consensus.cc:348] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } } peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } }
I20260811 11:55:58.255550 15490 raft_consensus.cc:374] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:58.255614 15490 raft_consensus.cc:729] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0042fe01318b4642937a2a7e5fc7f97f, State: Initialized, Role: FOLLOWER
I20260811 11:55:58.256101 15490 consensus_queue.cc:260] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 0042fe01318b4642937a2a7e5fc7f97f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } } peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } }
I20260811 11:55:58.257092 15490 ts_tablet_manager.cc:1428] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 0042fe01318b4642937a2a7e5fc7f97f: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 11:55:58.257467 15458 heartbeater.cc:503] Master 127.6.143.190:39159 was elected leader, sending a full tablet report...
I20260811 11:55:58.258047 15488 raft_consensus.cc:348] T c33c4c38e22c4e0d9df4a7ffc35168d6 P f0a368c2bba649cfa232d7321568e655 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } } peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } }
I20260811 11:55:58.258047 15489 raft_consensus.cc:348] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } } peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } }
I20260811 11:55:58.258291 15489 raft_consensus.cc:374] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:58.258291 15488 raft_consensus.cc:374] T c33c4c38e22c4e0d9df4a7ffc35168d6 P f0a368c2bba649cfa232d7321568e655 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:58.258350 15489 raft_consensus.cc:729] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87106c511d364f078ea5c1aa4a8be5dd, State: Initialized, Role: FOLLOWER
I20260811 11:55:58.258359 15488 raft_consensus.cc:729] T c33c4c38e22c4e0d9df4a7ffc35168d6 P f0a368c2bba649cfa232d7321568e655 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0a368c2bba649cfa232d7321568e655, State: Initialized, Role: FOLLOWER
I20260811 11:55:58.258837 15488 consensus_queue.cc:260] T c33c4c38e22c4e0d9df4a7ffc35168d6 P f0a368c2bba649cfa232d7321568e655 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } } peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } }
I20260811 11:55:58.258837 15489 consensus_queue.cc:260] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } } peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } }
I20260811 11:55:58.259438 15138 heartbeater.cc:503] Master 127.6.143.190:39159 was elected leader, sending a full tablet report...
I20260811 11:55:58.259770 15489 ts_tablet_manager.cc:1428] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 11:55:58.259770 15488 ts_tablet_manager.cc:1428] T c33c4c38e22c4e0d9df4a7ffc35168d6 P f0a368c2bba649cfa232d7321568e655: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 11:55:58.260084 15294 heartbeater.cc:503] Master 127.6.143.190:39159 was elected leader, sending a full tablet report...
I20260811 11:55:58.365227 15496 raft_consensus.cc:482] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:55:58.365487 15496 raft_consensus.cc:504] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } } peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } }
I20260811 11:55:58.366755 15496 leader_election.cc:288] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0042fe01318b4642937a2a7e5fc7f97f (127.6.143.131:46615), f0a368c2bba649cfa232d7321568e655 (127.6.143.129:40905)
I20260811 11:55:58.370630 15175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:58.366959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:46615 (local address 127.6.143.130:46679)
0811 11:55:58.367172 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:58.367189 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:58.367287 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:58.367552 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:58.367555 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:58.367572 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:58.367691 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:58.367697 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.368645 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.368651 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:58.369435 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:58.369444 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.370103 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.370107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:58.370270 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.370304 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 11:55:58.370421 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 11:55:58.370506 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133}
I20260811 11:55:58.370630 15483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:58.367163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:46679 (local address 127.6.143.131:46615)
0811 11:55:58.367284 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:58.367289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:58.367304 (+ 15us) server_negotiation.cc:407] Connection header received
0811 11:55:58.367390 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:58.367393 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:58.367444 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:58.367528 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:58.367812 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.368531 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.369568 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.369975 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.370104 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.370167 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:58.370420 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 11:55:58.370501 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 11:55:58.371169 15498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:58.367042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:40905 (local address 127.6.143.130:41811)
0811 11:55:58.367477 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:58.367488 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:58.367625 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:58.367947 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:58.367950 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:58.367968 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:58.368109 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:58.368117 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.368840 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.368846 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:58.369982 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:58.369992 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.370757 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.370761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:58.370861 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.370881 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 11:55:58.370992 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 11:55:58.371068 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":150,"threads_started":1}
I20260811 11:55:58.371217 15484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:58.367156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:41811 (local address 127.6.143.129:40905)
0811 11:55:58.367292 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:58.367297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:58.367586 (+ 289us) server_negotiation.cc:407] Connection header received
0811 11:55:58.367748 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:58.367756 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:58.367799 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:58.367880 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:58.368239 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.368728 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.370118 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.370625 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.370757 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.370805 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:58.371023 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 11:55:58.371106 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 11:55:58.371327 15401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c33c4c38e22c4e0d9df4a7ffc35168d6" candidate_uuid: "87106c511d364f078ea5c1aa4a8be5dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0042fe01318b4642937a2a7e5fc7f97f" is_pre_election: true
I20260811 11:55:58.371706 15401 raft_consensus.cc:2454] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87106c511d364f078ea5c1aa4a8be5dd in term 0.
I20260811 11:55:58.372107 15085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c33c4c38e22c4e0d9df4a7ffc35168d6" candidate_uuid: "87106c511d364f078ea5c1aa4a8be5dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0a368c2bba649cfa232d7321568e655" is_pre_election: true
I20260811 11:55:58.372269 15170 leader_election.cc:302] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0042fe01318b4642937a2a7e5fc7f97f, 87106c511d364f078ea5c1aa4a8be5dd; no voters:
I20260811 11:55:58.372433 15085 raft_consensus.cc:2454] T c33c4c38e22c4e0d9df4a7ffc35168d6 P f0a368c2bba649cfa232d7321568e655 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87106c511d364f078ea5c1aa4a8be5dd in term 0.
I20260811 11:55:58.372609 15496 raft_consensus.cc:2781] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:55:58.372713 15496 raft_consensus.cc:482] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:55:58.372771 15496 raft_consensus.cc:3037] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:58.373890 15496 raft_consensus.cc:504] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } } peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } }
I20260811 11:55:58.374233 15496 leader_election.cc:288] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd [CANDIDATE]: Term 1 election: Requested vote from peers 0042fe01318b4642937a2a7e5fc7f97f (127.6.143.131:46615), f0a368c2bba649cfa232d7321568e655 (127.6.143.129:40905)
I20260811 11:55:58.374598 15401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c33c4c38e22c4e0d9df4a7ffc35168d6" candidate_uuid: "87106c511d364f078ea5c1aa4a8be5dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0042fe01318b4642937a2a7e5fc7f97f"
I20260811 11:55:58.374603 15085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c33c4c38e22c4e0d9df4a7ffc35168d6" candidate_uuid: "87106c511d364f078ea5c1aa4a8be5dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0a368c2bba649cfa232d7321568e655"
I20260811 11:55:58.374722 15085 raft_consensus.cc:3037] T c33c4c38e22c4e0d9df4a7ffc35168d6 P f0a368c2bba649cfa232d7321568e655 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:58.374722 15401 raft_consensus.cc:3037] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:58.376080 15085 raft_consensus.cc:2454] T c33c4c38e22c4e0d9df4a7ffc35168d6 P f0a368c2bba649cfa232d7321568e655 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87106c511d364f078ea5c1aa4a8be5dd in term 1.
I20260811 11:55:58.376080 15401 raft_consensus.cc:2454] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 0042fe01318b4642937a2a7e5fc7f97f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87106c511d364f078ea5c1aa4a8be5dd in term 1.
I20260811 11:55:58.376368 15170 leader_election.cc:302] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0042fe01318b4642937a2a7e5fc7f97f, 87106c511d364f078ea5c1aa4a8be5dd; no voters:
I20260811 11:55:58.376559 15496 raft_consensus.cc:2781] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:58.376811 15496 raft_consensus.cc:686] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd [term 1 LEADER]: Becoming Leader. State: Replica: 87106c511d364f078ea5c1aa4a8be5dd, State: Running, Role: LEADER
I20260811 11:55:58.377148 15496 consensus_queue.cc:237] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } } peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } }
I20260811 11:55:58.380353 14910 catalog_manager.cc:5495] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 87106c511d364f078ea5c1aa4a8be5dd (127.6.143.130). New cstate: current_term: 1 leader_uuid: "87106c511d364f078ea5c1aa4a8be5dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } health_report { overall_health: HEALTHY } } }
W20260811 11:55:58.417902 15459 tablet.cc:2369] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 0042fe01318b4642937a2a7e5fc7f97f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:55:58.421072 15487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:58.417377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50476 (local address 127.6.143.130:37463)
0811 11:55:58.417522 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:58.417526 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:58.417541 (+ 15us) server_negotiation.cc:407] Connection header received
0811 11:55:58.417617 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:58.417620 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:58.417683 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:58.417797 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:55:58.418154 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.418756 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.419616 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.419840 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.420026 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.420301 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:55:58.420687 (+ 386us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:55:58.420813 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:58.420878 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 11:55:58.420939 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 11:55:58.427417 15401 raft_consensus.cc:1261] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 0042fe01318b4642937a2a7e5fc7f97f [term 1 FOLLOWER]: Refusing update from remote peer 87106c511d364f078ea5c1aa4a8be5dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:58.427412 15085 raft_consensus.cc:1261] T c33c4c38e22c4e0d9df4a7ffc35168d6 P f0a368c2bba649cfa232d7321568e655 [term 1 FOLLOWER]: Refusing update from remote peer 87106c511d364f078ea5c1aa4a8be5dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:58.428154 15496 consensus_queue.cc:1035] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:58.428339 15499 consensus_queue.cc:1035] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 11:55:58.434163 15295 tablet.cc:2369] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:55:58.438117 15504 mvcc.cc:205] Tried to move back new op lower bound from 7317296572111208448 to 7317296571915444224. Current Snapshot: MvccSnapshot[applied={T|T < 7317296572111208448}]
I20260811 11:55:58.438401 15503 mvcc.cc:205] Tried to move back new op lower bound from 7317296572111208448 to 7317296571915444224. Current Snapshot: MvccSnapshot[applied={T|T < 7317296572111208448}]
I20260811 11:55:58.443078 14909 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:55:58.443243 14909 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:55:58.445484 14909 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f9ce963fd5964673929ebfb3cfbf4f27: Sending DeleteTablet for 3 replicas of tablet c33c4c38e22c4e0d9df4a7ffc35168d6
I20260811 11:55:58.446341 15063 tablet_service.cc:1514] Processing DeleteTablet for tablet c33c4c38e22c4e0d9df4a7ffc35168d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:58 UTC) from {username='oryx', principal='oryx/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:39422
I20260811 11:55:58.446338 15380 tablet_service.cc:1514] Processing DeleteTablet for tablet c33c4c38e22c4e0d9df4a7ffc35168d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:58 UTC) from {username='oryx', principal='oryx/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:50706
I20260811 11:55:58.446841 15511 tablet_replica.cc:330] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 0042fe01318b4642937a2a7e5fc7f97f: stopping tablet replica
I20260811 11:55:58.446940 15222 tablet_service.cc:1514] Processing DeleteTablet for tablet c33c4c38e22c4e0d9df4a7ffc35168d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:58 UTC) from {username='oryx', principal='oryx/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:50462
I20260811 11:55:58.447081 14909 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55196:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 11:55:58.447359 15513 tablet_replica.cc:330] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd: stopping tablet replica
W20260811 11:55:58.447481 14909 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:55:58.447479 15511 raft_consensus.cc:2229] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 0042fe01318b4642937a2a7e5fc7f97f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:55:58.447636 15513 raft_consensus.cc:2229] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:55:58.447953 15511 raft_consensus.cc:2258] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 0042fe01318b4642937a2a7e5fc7f97f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:58.448012 15513 raft_consensus.cc:2258] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:58.448343 15512 tablet_replica.cc:330] T c33c4c38e22c4e0d9df4a7ffc35168d6 P f0a368c2bba649cfa232d7321568e655: stopping tablet replica
I20260811 11:55:58.448608 15512 raft_consensus.cc:2229] T c33c4c38e22c4e0d9df4a7ffc35168d6 P f0a368c2bba649cfa232d7321568e655 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:55:58.448897 15512 raft_consensus.cc:2258] T c33c4c38e22c4e0d9df4a7ffc35168d6 P f0a368c2bba649cfa232d7321568e655 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:58.449739 15511 ts_tablet_manager.cc:1905] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 0042fe01318b4642937a2a7e5fc7f97f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:58.450856 15513 ts_tablet_manager.cc:1905] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:58.452749 15511 ts_tablet_manager.cc:1918] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 0042fe01318b4642937a2a7e5fc7f97f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:55:58.452919 15511 log.cc:1196] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 0042fe01318b4642937a2a7e5fc7f97f: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-2/wal/wals/c33c4c38e22c4e0d9df4a7ffc35168d6
I20260811 11:55:58.453380 15511 ts_tablet_manager.cc:1939] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 0042fe01318b4642937a2a7e5fc7f97f: Deleting consensus metadata
I20260811 11:55:58.453550 15513 ts_tablet_manager.cc:1918] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:55:58.453646 15513 log.cc:1196] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-1/wal/wals/c33c4c38e22c4e0d9df4a7ffc35168d6
I20260811 11:55:58.453975 15513 ts_tablet_manager.cc:1939] T c33c4c38e22c4e0d9df4a7ffc35168d6 P 87106c511d364f078ea5c1aa4a8be5dd: Deleting consensus metadata
I20260811 11:55:58.454885 15063 tablet_service.cc:1467] Processing CreateTablet for tablet f574bc207ebd4605b1940daf4e8d7eb1 (DEFAULT_TABLE table=test-table [id=5de9d9aabd9944289b74a2309a52725d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:58.455020 14896 catalog_manager.cc:4841] TS 87106c511d364f078ea5c1aa4a8be5dd (127.6.143.130:37463): tablet c33c4c38e22c4e0d9df4a7ffc35168d6 (table test-table [id=4961d3c520cf4fa286ab4aaaf05f0868]) successfully deleted
I20260811 11:55:58.455193 15063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f574bc207ebd4605b1940daf4e8d7eb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:58.456148 15380 tablet_service.cc:1467] Processing CreateTablet for tablet f574bc207ebd4605b1940daf4e8d7eb1 (DEFAULT_TABLE table=test-table [id=5de9d9aabd9944289b74a2309a52725d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:58.456382 15380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f574bc207ebd4605b1940daf4e8d7eb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:58.457297 15488 tablet_bootstrap.cc:492] T f574bc207ebd4605b1940daf4e8d7eb1 P f0a368c2bba649cfa232d7321568e655: Bootstrap starting.
I20260811 11:55:58.458266 15488 tablet_bootstrap.cc:654] T f574bc207ebd4605b1940daf4e8d7eb1 P f0a368c2bba649cfa232d7321568e655: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:58.458478 15222 tablet_service.cc:1467] Processing CreateTablet for tablet f574bc207ebd4605b1940daf4e8d7eb1 (DEFAULT_TABLE table=test-table [id=5de9d9aabd9944289b74a2309a52725d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:58.458705 15222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f574bc207ebd4605b1940daf4e8d7eb1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:58.459439 15488 tablet_bootstrap.cc:492] T f574bc207ebd4605b1940daf4e8d7eb1 P f0a368c2bba649cfa232d7321568e655: No bootstrap required, opened a new log
I20260811 11:55:58.459544 15488 ts_tablet_manager.cc:1397] T f574bc207ebd4605b1940daf4e8d7eb1 P f0a368c2bba649cfa232d7321568e655: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:58.460028 15488 raft_consensus.cc:348] T f574bc207ebd4605b1940daf4e8d7eb1 P f0a368c2bba649cfa232d7321568e655 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:58.460139 15488 raft_consensus.cc:374] T f574bc207ebd4605b1940daf4e8d7eb1 P f0a368c2bba649cfa232d7321568e655 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:58.460182 15488 raft_consensus.cc:729] T f574bc207ebd4605b1940daf4e8d7eb1 P f0a368c2bba649cfa232d7321568e655 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0a368c2bba649cfa232d7321568e655, State: Initialized, Role: FOLLOWER
I20260811 11:55:58.460309 15488 consensus_queue.cc:260] T f574bc207ebd4605b1940daf4e8d7eb1 P f0a368c2bba649cfa232d7321568e655 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:58.460583 15488 ts_tablet_manager.cc:1428] T f574bc207ebd4605b1940daf4e8d7eb1 P f0a368c2bba649cfa232d7321568e655: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 11:55:58.460876 15489 tablet_bootstrap.cc:492] T f574bc207ebd4605b1940daf4e8d7eb1 P 87106c511d364f078ea5c1aa4a8be5dd: Bootstrap starting.
I20260811 11:55:58.461756 15489 tablet_bootstrap.cc:654] T f574bc207ebd4605b1940daf4e8d7eb1 P 87106c511d364f078ea5c1aa4a8be5dd: Neither blocks nor log segments found. Creating new log.
W20260811 11:55:58.462083 15139 tablet.cc:2369] T f574bc207ebd4605b1940daf4e8d7eb1 P f0a368c2bba649cfa232d7321568e655: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:55:58.462383 14898 catalog_manager.cc:4841] TS 0042fe01318b4642937a2a7e5fc7f97f (127.6.143.131:46615): tablet c33c4c38e22c4e0d9df4a7ffc35168d6 (table test-table [id=4961d3c520cf4fa286ab4aaaf05f0868]) successfully deleted
I20260811 11:55:58.462913 15490 tablet_bootstrap.cc:492] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f: Bootstrap starting.
I20260811 11:55:58.463793 15490 tablet_bootstrap.cc:654] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:58.463918 15512 ts_tablet_manager.cc:1905] T c33c4c38e22c4e0d9df4a7ffc35168d6 P f0a368c2bba649cfa232d7321568e655: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:58.464716 15489 tablet_bootstrap.cc:492] T f574bc207ebd4605b1940daf4e8d7eb1 P 87106c511d364f078ea5c1aa4a8be5dd: No bootstrap required, opened a new log
I20260811 11:55:58.464814 15489 ts_tablet_manager.cc:1397] T f574bc207ebd4605b1940daf4e8d7eb1 P 87106c511d364f078ea5c1aa4a8be5dd: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260811 11:55:58.465236 15490 tablet_bootstrap.cc:492] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f: No bootstrap required, opened a new log
I20260811 11:55:58.465315 15490 ts_tablet_manager.cc:1397] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:58.465287 15489 raft_consensus.cc:348] T f574bc207ebd4605b1940daf4e8d7eb1 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:58.465410 15489 raft_consensus.cc:374] T f574bc207ebd4605b1940daf4e8d7eb1 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:58.465448 15489 raft_consensus.cc:729] T f574bc207ebd4605b1940daf4e8d7eb1 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87106c511d364f078ea5c1aa4a8be5dd, State: Initialized, Role: FOLLOWER
I20260811 11:55:58.465603 15489 consensus_queue.cc:260] T f574bc207ebd4605b1940daf4e8d7eb1 P 87106c511d364f078ea5c1aa4a8be5dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:58.465848 15489 ts_tablet_manager.cc:1428] T f574bc207ebd4605b1940daf4e8d7eb1 P 87106c511d364f078ea5c1aa4a8be5dd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:55:58.465802 15490 raft_consensus.cc:348] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:58.465914 15490 raft_consensus.cc:374] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:58.465947 15490 raft_consensus.cc:729] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0042fe01318b4642937a2a7e5fc7f97f, State: Initialized, Role: FOLLOWER
I20260811 11:55:58.466073 15490 consensus_queue.cc:260] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:58.466334 15490 ts_tablet_manager.cc:1428] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:55:58.466830 15512 ts_tablet_manager.cc:1918] T c33c4c38e22c4e0d9df4a7ffc35168d6 P f0a368c2bba649cfa232d7321568e655: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:55:58.466913 15512 log.cc:1196] T c33c4c38e22c4e0d9df4a7ffc35168d6 P f0a368c2bba649cfa232d7321568e655: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-0/wal/wals/c33c4c38e22c4e0d9df4a7ffc35168d6
I20260811 11:55:58.467199 15512 ts_tablet_manager.cc:1939] T c33c4c38e22c4e0d9df4a7ffc35168d6 P f0a368c2bba649cfa232d7321568e655: Deleting consensus metadata
I20260811 11:55:58.468290 14896 catalog_manager.cc:4841] TS f0a368c2bba649cfa232d7321568e655 (127.6.143.129:40905): tablet c33c4c38e22c4e0d9df4a7ffc35168d6 (table test-table [id=4961d3c520cf4fa286ab4aaaf05f0868]) successfully deleted
I20260811 11:55:58.592232 15494 raft_consensus.cc:482] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:55:58.592411 15494 raft_consensus.cc:504] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:58.593652 15494 leader_election.cc:288] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0a368c2bba649cfa232d7321568e655 (127.6.143.129:40905), 87106c511d364f078ea5c1aa4a8be5dd (127.6.143.130:37463)
I20260811 11:55:58.597776 15336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:58.593929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:40905 (local address 127.6.143.131:34445)
0811 11:55:58.594212 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:58.594228 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:58.594339 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:58.594729 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:58.594734 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:58.594755 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:58.594914 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:58.594920 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.595780 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.595785 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:58.596676 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:58.596686 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.597322 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.597326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:58.597440 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.597468 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 11:55:58.597557 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 11:55:58.597628 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":21}
I20260811 11:55:58.597776 15484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:58.594058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:34445 (local address 127.6.143.129:40905)
0811 11:55:58.594217 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:58.594222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:58.594487 (+ 265us) server_negotiation.cc:407] Connection header received
0811 11:55:58.594548 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:58.594551 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:58.594599 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:58.594686 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:58.595033 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.595652 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.596813 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.597199 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.597333 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.597401 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:58.597574 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 11:55:58.597647 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 11:55:58.598063 15487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:58.594417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:58861 (local address 127.6.143.130:37463)
0811 11:55:58.594559 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:58.594564 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:58.594709 (+ 145us) server_negotiation.cc:407] Connection header received
0811 11:55:58.594890 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:58.594893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:58.594948 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:58.595022 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:55:58.595325 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.596062 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.597107 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.597495 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.597630 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.597681 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:58.597889 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 11:55:58.597958 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 11:55:58.598088 15519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:58.594225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:37463 (local address 127.6.143.131:58861)
0811 11:55:58.594587 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:58.594602 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:58.594766 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:58.595062 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:58.595066 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:58.595084 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:55:58.595199 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:58.595204 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.596166 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.596171 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:58.596974 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:58.596984 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.597636 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.597640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:58.597748 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.597770 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 11:55:58.597872 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 11:55:58.597939 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":15,"thread_start_us":86,"threads_started":1}
I20260811 11:55:58.598330 15085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f574bc207ebd4605b1940daf4e8d7eb1" candidate_uuid: "0042fe01318b4642937a2a7e5fc7f97f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0a368c2bba649cfa232d7321568e655" is_pre_election: true
I20260811 11:55:58.598515 15085 raft_consensus.cc:2454] T f574bc207ebd4605b1940daf4e8d7eb1 P f0a368c2bba649cfa232d7321568e655 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0042fe01318b4642937a2a7e5fc7f97f in term 0.
I20260811 11:55:58.598654 15244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f574bc207ebd4605b1940daf4e8d7eb1" candidate_uuid: "0042fe01318b4642937a2a7e5fc7f97f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87106c511d364f078ea5c1aa4a8be5dd" is_pre_election: true
I20260811 11:55:58.598906 15244 raft_consensus.cc:2454] T f574bc207ebd4605b1940daf4e8d7eb1 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0042fe01318b4642937a2a7e5fc7f97f in term 0.
I20260811 11:55:58.598893 15326 leader_election.cc:302] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0042fe01318b4642937a2a7e5fc7f97f, f0a368c2bba649cfa232d7321568e655; no voters:
I20260811 11:55:58.599174 15494 raft_consensus.cc:2781] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:55:58.599238 15494 raft_consensus.cc:482] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:55:58.599278 15494 raft_consensus.cc:3037] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:58.600353 15494 raft_consensus.cc:504] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:58.600678 15494 leader_election.cc:288] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f [CANDIDATE]: Term 1 election: Requested vote from peers f0a368c2bba649cfa232d7321568e655 (127.6.143.129:40905), 87106c511d364f078ea5c1aa4a8be5dd (127.6.143.130:37463)
I20260811 11:55:58.601052 15085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f574bc207ebd4605b1940daf4e8d7eb1" candidate_uuid: "0042fe01318b4642937a2a7e5fc7f97f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0a368c2bba649cfa232d7321568e655"
I20260811 11:55:58.601058 15244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f574bc207ebd4605b1940daf4e8d7eb1" candidate_uuid: "0042fe01318b4642937a2a7e5fc7f97f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87106c511d364f078ea5c1aa4a8be5dd"
I20260811 11:55:58.601179 15244 raft_consensus.cc:3037] T f574bc207ebd4605b1940daf4e8d7eb1 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:58.601179 15085 raft_consensus.cc:3037] T f574bc207ebd4605b1940daf4e8d7eb1 P f0a368c2bba649cfa232d7321568e655 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:58.602295 15244 raft_consensus.cc:2454] T f574bc207ebd4605b1940daf4e8d7eb1 P 87106c511d364f078ea5c1aa4a8be5dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0042fe01318b4642937a2a7e5fc7f97f in term 1.
I20260811 11:55:58.602295 15085 raft_consensus.cc:2454] T f574bc207ebd4605b1940daf4e8d7eb1 P f0a368c2bba649cfa232d7321568e655 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0042fe01318b4642937a2a7e5fc7f97f in term 1.
I20260811 11:55:58.602584 15326 leader_election.cc:302] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0042fe01318b4642937a2a7e5fc7f97f, 87106c511d364f078ea5c1aa4a8be5dd; no voters:
I20260811 11:55:58.602743 15494 raft_consensus.cc:2781] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:58.602954 15494 raft_consensus.cc:686] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f [term 1 LEADER]: Becoming Leader. State: Replica: 0042fe01318b4642937a2a7e5fc7f97f, State: Running, Role: LEADER
I20260811 11:55:58.603200 15494 consensus_queue.cc:237] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:58.605521 14910 catalog_manager.cc:5495] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0042fe01318b4642937a2a7e5fc7f97f (127.6.143.131). New cstate: current_term: 1 leader_uuid: "0042fe01318b4642937a2a7e5fc7f97f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } health_report { overall_health: HEALTHY } } }
Aug 11 11:55:58 dist-test-slave-zkpd krb5kdc[14860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449357, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.190@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM
I20260811 11:55:58.674126 15530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:58.665810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39159 (local address 127.0.0.1:55202)
0811 11:55:58.666102 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:58.666114 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:58.666257 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:58.666512 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:58.666515 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:58.666989 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:55:58.667080 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:58.667086 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.667828 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.667834 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:58.668448 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:58.668457 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.668919 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.668922 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:58.669052 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.669577 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:55:58.669589 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:55:58.671065 (+ 1476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:55:58.672549 (+ 1484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:58.672561 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:58.672567 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:58.672788 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:58.673068 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:55:58.673071 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:55:58.673073 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:55:58.673152 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:55:58.673512 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:55:58.673520 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:55:58.673628 (+ 108us) client_negotiation.cc:769] Sending connection context
0811 11:55:58.673773 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 11:55:58.673922 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":34,"thread_start_us":108,"threads_started":1}
I20260811 11:55:58.674126 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:58.665836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55202 (local address 127.6.143.190:39159)
0811 11:55:58.666024 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:58.666030 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:58.666194 (+ 164us) server_negotiation.cc:407] Connection header received
0811 11:55:58.666356 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:58.666360 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:58.666399 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:58.666493 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:58.667197 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.667722 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.668583 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.668807 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.668922 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.671198 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:58.671202 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:58.671205 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:58.671234 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:58.672447 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:58.672886 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:58.672891 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:58.672893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:58.672972 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:58.673243 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:58.673246 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:58.673248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:58.673409 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:58.673512 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:58.673835 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 11:55:58.673971 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":94}
I20260811 11:55:58.677958 14910 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:55202:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 11:55:58.678714 14910 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:55:58.683733 15063 tablet_service.cc:1467] Processing CreateTablet for tablet a3095daa8ecf4ada8b4c1ddae96598b5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=15e5033d1fc84253bdc5bb562d70caad]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 11:55:58.683772 15222 tablet_service.cc:1467] Processing CreateTablet for tablet a3095daa8ecf4ada8b4c1ddae96598b5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=15e5033d1fc84253bdc5bb562d70caad]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 11:55:58.683882 15380 tablet_service.cc:1467] Processing CreateTablet for tablet a3095daa8ecf4ada8b4c1ddae96598b5 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=15e5033d1fc84253bdc5bb562d70caad]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 11:55:58.684214 15222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3095daa8ecf4ada8b4c1ddae96598b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:58.684216 15063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3095daa8ecf4ada8b4c1ddae96598b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:58.684262 15380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3095daa8ecf4ada8b4c1ddae96598b5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:58.686674 15489 tablet_bootstrap.cc:492] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd: Bootstrap starting.
I20260811 11:55:58.687140 15490 tablet_bootstrap.cc:492] T a3095daa8ecf4ada8b4c1ddae96598b5 P 0042fe01318b4642937a2a7e5fc7f97f: Bootstrap starting.
I20260811 11:55:58.687590 15489 tablet_bootstrap.cc:654] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:58.688090 15490 tablet_bootstrap.cc:654] T a3095daa8ecf4ada8b4c1ddae96598b5 P 0042fe01318b4642937a2a7e5fc7f97f: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:58.688470 15488 tablet_bootstrap.cc:492] T a3095daa8ecf4ada8b4c1ddae96598b5 P f0a368c2bba649cfa232d7321568e655: Bootstrap starting.
I20260811 11:55:58.689343 15490 tablet_bootstrap.cc:492] T a3095daa8ecf4ada8b4c1ddae96598b5 P 0042fe01318b4642937a2a7e5fc7f97f: No bootstrap required, opened a new log
I20260811 11:55:58.689460 15490 ts_tablet_manager.cc:1397] T a3095daa8ecf4ada8b4c1ddae96598b5 P 0042fe01318b4642937a2a7e5fc7f97f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:58.689476 15488 tablet_bootstrap.cc:654] T a3095daa8ecf4ada8b4c1ddae96598b5 P f0a368c2bba649cfa232d7321568e655: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:58.690076 15490 raft_consensus.cc:348] T a3095daa8ecf4ada8b4c1ddae96598b5 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:58.690210 15490 raft_consensus.cc:374] T a3095daa8ecf4ada8b4c1ddae96598b5 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:58.690245 15490 raft_consensus.cc:729] T a3095daa8ecf4ada8b4c1ddae96598b5 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0042fe01318b4642937a2a7e5fc7f97f, State: Initialized, Role: FOLLOWER
I20260811 11:55:58.690371 15490 consensus_queue.cc:260] T a3095daa8ecf4ada8b4c1ddae96598b5 P 0042fe01318b4642937a2a7e5fc7f97f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:58.690439 15489 tablet_bootstrap.cc:492] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd: No bootstrap required, opened a new log
I20260811 11:55:58.690527 15489 ts_tablet_manager.cc:1397] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 11:55:58.690629 15494 tablet_replica.cc:439] T a3095daa8ecf4ada8b4c1ddae96598b5 P 0042fe01318b4642937a2a7e5fc7f97f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } } }
I20260811 11:55:58.690753 15494 tablet_replica.cc:442] T a3095daa8ecf4ada8b4c1ddae96598b5 P 0042fe01318b4642937a2a7e5fc7f97f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:55:58.690953 15489 raft_consensus.cc:348] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:58.691063 15489 raft_consensus.cc:374] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:58.691094 15489 raft_consensus.cc:729] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87106c511d364f078ea5c1aa4a8be5dd, State: Initialized, Role: FOLLOWER
I20260811 11:55:58.691227 15489 consensus_queue.cc:260] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:58.691423 15490 ts_tablet_manager.cc:1428] T a3095daa8ecf4ada8b4c1ddae96598b5 P 0042fe01318b4642937a2a7e5fc7f97f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:58.691497 15489 ts_tablet_manager.cc:1428] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:55:58.691690 15496 tablet_replica.cc:439] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } } }
I20260811 11:55:58.691835 15496 tablet_replica.cc:442] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:55:58.692399 15488 tablet_bootstrap.cc:492] T a3095daa8ecf4ada8b4c1ddae96598b5 P f0a368c2bba649cfa232d7321568e655: No bootstrap required, opened a new log
I20260811 11:55:58.692510 15488 ts_tablet_manager.cc:1397] T a3095daa8ecf4ada8b4c1ddae96598b5 P f0a368c2bba649cfa232d7321568e655: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 11:55:58.692921 15488 raft_consensus.cc:348] T a3095daa8ecf4ada8b4c1ddae96598b5 P f0a368c2bba649cfa232d7321568e655 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:58.693065 15488 raft_consensus.cc:374] T a3095daa8ecf4ada8b4c1ddae96598b5 P f0a368c2bba649cfa232d7321568e655 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:58.693107 15488 raft_consensus.cc:729] T a3095daa8ecf4ada8b4c1ddae96598b5 P f0a368c2bba649cfa232d7321568e655 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0a368c2bba649cfa232d7321568e655, State: Initialized, Role: FOLLOWER
I20260811 11:55:58.693244 15488 consensus_queue.cc:260] T a3095daa8ecf4ada8b4c1ddae96598b5 P f0a368c2bba649cfa232d7321568e655 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:58.693764 15488 ts_tablet_manager.cc:1428] T a3095daa8ecf4ada8b4c1ddae96598b5 P f0a368c2bba649cfa232d7321568e655: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:55:58.693712 15495 tablet_replica.cc:439] T a3095daa8ecf4ada8b4c1ddae96598b5 P f0a368c2bba649cfa232d7321568e655: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } } }
I20260811 11:55:58.693845 15495 tablet_replica.cc:442] T a3095daa8ecf4ada8b4c1ddae96598b5 P f0a368c2bba649cfa232d7321568e655: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:55:58.787015 15496 raft_consensus.cc:482] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:55:58.787153 15496 raft_consensus.cc:504] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:58.787535 15496 leader_election.cc:288] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0a368c2bba649cfa232d7321568e655 (127.6.143.129:40905), 0042fe01318b4642937a2a7e5fc7f97f (127.6.143.131:46615)
I20260811 11:55:58.787998 15085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3095daa8ecf4ada8b4c1ddae96598b5" candidate_uuid: "87106c511d364f078ea5c1aa4a8be5dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0a368c2bba649cfa232d7321568e655" is_pre_election: true
I20260811 11:55:58.788007 15401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3095daa8ecf4ada8b4c1ddae96598b5" candidate_uuid: "87106c511d364f078ea5c1aa4a8be5dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0042fe01318b4642937a2a7e5fc7f97f" is_pre_election: true
I20260811 11:55:58.788163 15085 raft_consensus.cc:2454] T a3095daa8ecf4ada8b4c1ddae96598b5 P f0a368c2bba649cfa232d7321568e655 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87106c511d364f078ea5c1aa4a8be5dd in term 0.
I20260811 11:55:58.788175 15401 raft_consensus.cc:2454] T a3095daa8ecf4ada8b4c1ddae96598b5 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87106c511d364f078ea5c1aa4a8be5dd in term 0.
I20260811 11:55:58.788501 15167 leader_election.cc:302] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87106c511d364f078ea5c1aa4a8be5dd, f0a368c2bba649cfa232d7321568e655; no voters:
I20260811 11:55:58.788702 15496 raft_consensus.cc:2781] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:55:58.788765 15496 raft_consensus.cc:482] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:55:58.788793 15496 raft_consensus.cc:3037] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:58.789731 15496 raft_consensus.cc:504] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:58.790060 15496 leader_election.cc:288] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd [CANDIDATE]: Term 1 election: Requested vote from peers f0a368c2bba649cfa232d7321568e655 (127.6.143.129:40905), 0042fe01318b4642937a2a7e5fc7f97f (127.6.143.131:46615)
I20260811 11:55:58.790412 15085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3095daa8ecf4ada8b4c1ddae96598b5" candidate_uuid: "87106c511d364f078ea5c1aa4a8be5dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0a368c2bba649cfa232d7321568e655"
I20260811 11:55:58.790524 15085 raft_consensus.cc:3037] T a3095daa8ecf4ada8b4c1ddae96598b5 P f0a368c2bba649cfa232d7321568e655 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:58.790465 15401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3095daa8ecf4ada8b4c1ddae96598b5" candidate_uuid: "87106c511d364f078ea5c1aa4a8be5dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0042fe01318b4642937a2a7e5fc7f97f"
I20260811 11:55:58.790611 15401 raft_consensus.cc:3037] T a3095daa8ecf4ada8b4c1ddae96598b5 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:58.791421 15085 raft_consensus.cc:2454] T a3095daa8ecf4ada8b4c1ddae96598b5 P f0a368c2bba649cfa232d7321568e655 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87106c511d364f078ea5c1aa4a8be5dd in term 1.
I20260811 11:55:58.791615 15401 raft_consensus.cc:2454] T a3095daa8ecf4ada8b4c1ddae96598b5 P 0042fe01318b4642937a2a7e5fc7f97f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87106c511d364f078ea5c1aa4a8be5dd in term 1.
I20260811 11:55:58.791703 15167 leader_election.cc:302] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87106c511d364f078ea5c1aa4a8be5dd, f0a368c2bba649cfa232d7321568e655; no voters:
I20260811 11:55:58.792006 15496 raft_consensus.cc:2781] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:58.792105 15496 raft_consensus.cc:686] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd [term 1 LEADER]: Becoming Leader. State: Replica: 87106c511d364f078ea5c1aa4a8be5dd, State: Running, Role: LEADER
I20260811 11:55:58.792240 15496 consensus_queue.cc:237] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:58.792827 15499 tablet_replica.cc:439] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd: TxnStatusTablet state changed. Reason: New leader 87106c511d364f078ea5c1aa4a8be5dd. Latest consensus state: current_term: 1 leader_uuid: "87106c511d364f078ea5c1aa4a8be5dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } } }
I20260811 11:55:58.792948 15499 tablet_replica.cc:442] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd: This TxnStatusTablet replica's current role is: LEADER
I20260811 11:55:58.793183 15535 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 11:55:58.793354 15535 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 11:55:58.793936 14909 catalog_manager.cc:5495] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 87106c511d364f078ea5c1aa4a8be5dd (127.6.143.130). New cstate: current_term: 1 leader_uuid: "87106c511d364f078ea5c1aa4a8be5dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } health_report { overall_health: UNKNOWN } } }
I20260811 11:55:58.816920 15530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:58.813218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:37463 (local address 127.0.0.1:50488)
0811 11:55:58.813358 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:58.813376 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:58.813479 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:58.813773 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:58.813775 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:58.813781 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 11:55:58.813871 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:58.813876 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.814920 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.814922 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:58.815656 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:58.815666 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.816260 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.816263 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:58.816328 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.816375 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 11:55:58.816676 (+ 301us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 11:55:58.816689 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 11:55:58.816739 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 11:55:58.816808 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 11:55:58.816952 15487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:58.813310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50488 (local address 127.6.143.130:37463)
0811 11:55:58.813443 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:58.813447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:58.813463 (+ 16us) server_negotiation.cc:407] Connection header received
0811 11:55:58.813597 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:58.813601 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:58.813651 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:58.813748 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:55:58.813980 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.814786 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.815769 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.816158 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.816262 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.816512 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:55:58.816593 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:55:58.816666 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:58.816775 (+ 109us) server_negotiation.cc:299] Negotiation successful
0811 11:55:58.816827 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 11:55:58.819396 15085 raft_consensus.cc:1261] T a3095daa8ecf4ada8b4c1ddae96598b5 P f0a368c2bba649cfa232d7321568e655 [term 1 FOLLOWER]: Refusing update from remote peer 87106c511d364f078ea5c1aa4a8be5dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:58.819432 15401 raft_consensus.cc:1261] T a3095daa8ecf4ada8b4c1ddae96598b5 P 0042fe01318b4642937a2a7e5fc7f97f [term 1 FOLLOWER]: Refusing update from remote peer 87106c511d364f078ea5c1aa4a8be5dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:58.819856 15499 consensus_queue.cc:1035] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:58.820361 15499 consensus_queue.cc:1035] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:58.821718 15495 tablet_replica.cc:439] T a3095daa8ecf4ada8b4c1ddae96598b5 P f0a368c2bba649cfa232d7321568e655: TxnStatusTablet state changed. Reason: New leader 87106c511d364f078ea5c1aa4a8be5dd. Latest consensus state: current_term: 1 leader_uuid: "87106c511d364f078ea5c1aa4a8be5dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } } }
I20260811 11:55:58.821830 15495 tablet_replica.cc:442] T a3095daa8ecf4ada8b4c1ddae96598b5 P f0a368c2bba649cfa232d7321568e655: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:55:58.822083 15494 tablet_replica.cc:439] T a3095daa8ecf4ada8b4c1ddae96598b5 P 0042fe01318b4642937a2a7e5fc7f97f: TxnStatusTablet state changed. Reason: New leader 87106c511d364f078ea5c1aa4a8be5dd. Latest consensus state: current_term: 1 leader_uuid: "87106c511d364f078ea5c1aa4a8be5dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } } }
I20260811 11:55:58.822190 15494 tablet_replica.cc:442] T a3095daa8ecf4ada8b4c1ddae96598b5 P 0042fe01318b4642937a2a7e5fc7f97f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:55:58.823936 15496 tablet_replica.cc:439] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "87106c511d364f078ea5c1aa4a8be5dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } } }
I20260811 11:55:58.824047 15496 tablet_replica.cc:442] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd: This TxnStatusTablet replica's current role is: LEADER
I20260811 11:55:58.824149 15502 mvcc.cc:205] Tried to move back new op lower bound from 7317296573719990272 to 7317296573614088192. Current Snapshot: MvccSnapshot[applied={T|T < 7317296573719990272 or (T in {7317296573719990272})}]
I20260811 11:55:58.824616 15495 tablet_replica.cc:439] T a3095daa8ecf4ada8b4c1ddae96598b5 P f0a368c2bba649cfa232d7321568e655: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "87106c511d364f078ea5c1aa4a8be5dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } } }
I20260811 11:55:58.824779 15495 tablet_replica.cc:442] T a3095daa8ecf4ada8b4c1ddae96598b5 P f0a368c2bba649cfa232d7321568e655: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:55:58.826184 15496 tablet_replica.cc:439] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "87106c511d364f078ea5c1aa4a8be5dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } } }
I20260811 11:55:58.826296 15496 tablet_replica.cc:442] T a3095daa8ecf4ada8b4c1ddae96598b5 P 87106c511d364f078ea5c1aa4a8be5dd: This TxnStatusTablet replica's current role is: LEADER
I20260811 11:55:58.827145 15494 tablet_replica.cc:439] T a3095daa8ecf4ada8b4c1ddae96598b5 P 0042fe01318b4642937a2a7e5fc7f97f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "87106c511d364f078ea5c1aa4a8be5dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } } }
I20260811 11:55:58.827265 15494 tablet_replica.cc:442] T a3095daa8ecf4ada8b4c1ddae96598b5 P 0042fe01318b4642937a2a7e5fc7f97f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 11:55:58.834381 15483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:58.830360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50708 (local address 127.6.143.131:46615)
0811 11:55:58.830478 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:58.830488 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:58.830530 (+ 42us) server_negotiation.cc:407] Connection header received
0811 11:55:58.830717 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:58.830719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:58.830770 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:58.830857 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:55:58.831233 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.831737 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.832967 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.833172 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.833250 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.833529 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:55:58.833980 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:55:58.834072 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:58.834173 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 11:55:58.834217 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 11:55:58.845678 15487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:58.841478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:45655 (local address 127.6.143.130:37463)
0811 11:55:58.841599 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:58.841603 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:58.841837 (+ 234us) server_negotiation.cc:407] Connection header received
0811 11:55:58.841946 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:58.841948 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:58.841988 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:58.842050 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:55:58.842813 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.843423 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.844691 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.844895 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.844982 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.845306 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:55:58.845368 (+ 62us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:55:58.845421 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:58.845537 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 11:55:58.845581 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 11:55:58.845763 15545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:58.841382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:37463 (local address 127.6.143.131:45655)
0811 11:55:58.841752 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:58.841767 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:58.841870 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:58.842084 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:58.842087 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:58.842095 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 11:55:58.842688 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:58.842698 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.843544 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.843548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:58.844555 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:58.844567 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.845010 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.845013 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:58.845113 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.845165 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 11:55:58.845446 (+ 281us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 11:55:58.845453 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 11:55:58.845519 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 11:55:58.845593 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":105,"threads_started":1}
I20260811 11:55:58.852353 15085 raft_consensus.cc:1261] T f574bc207ebd4605b1940daf4e8d7eb1 P f0a368c2bba649cfa232d7321568e655 [term 1 FOLLOWER]: Refusing update from remote peer 0042fe01318b4642937a2a7e5fc7f97f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:58.852388 15244 raft_consensus.cc:1261] T f574bc207ebd4605b1940daf4e8d7eb1 P 87106c511d364f078ea5c1aa4a8be5dd [term 1 FOLLOWER]: Refusing update from remote peer 0042fe01318b4642937a2a7e5fc7f97f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:58.852855 15522 consensus_queue.cc:1035] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:58.853039 15494 consensus_queue.cc:1035] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f [LEADER]: Connected to new peer: Peer: permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:58.877430 15483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:58.873856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:56195 (local address 127.6.143.131:46615)
0811 11:55:58.873964 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:58.873969 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:58.874221 (+ 252us) server_negotiation.cc:407] Connection header received
0811 11:55:58.874364 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:58.874367 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:58.874418 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:58.874493 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:55:58.874734 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.875230 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.876202 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.876462 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.876521 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.876906 (+ 385us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:55:58.877019 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:55:58.877092 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:58.877241 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 11:55:58.877296 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 11:55:58.877460 15551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:58.873816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:46615 (local address 127.6.143.130:56195)
0811 11:55:58.874133 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:58.874146 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:58.874266 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:58.874517 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:58.874520 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:58.874529 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 11:55:58.874626 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:58.874638 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.875339 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.875345 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:58.876061 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:58.876077 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.876642 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.876646 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:58.876725 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.876777 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 11:55:58.877098 (+ 321us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 11:55:58.877105 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 11:55:58.877217 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 11:55:58.877296 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":86,"threads_started":1}
I20260811 11:55:58.902834 14909 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:55:58.903014 14909 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:55:58.905412 14909 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f9ce963fd5964673929ebfb3cfbf4f27: Sending DeleteTablet for 3 replicas of tablet f574bc207ebd4605b1940daf4e8d7eb1
I20260811 11:55:58.906121 15380 tablet_service.cc:1514] Processing DeleteTablet for tablet f574bc207ebd4605b1940daf4e8d7eb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:58 UTC) from {username='oryx', principal='oryx/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:50706
I20260811 11:55:58.906152 15063 tablet_service.cc:1514] Processing DeleteTablet for tablet f574bc207ebd4605b1940daf4e8d7eb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:58 UTC) from {username='oryx', principal='oryx/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:39422
I20260811 11:55:58.906306 15511 tablet_replica.cc:330] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f: stopping tablet replica
I20260811 11:55:58.906371 15512 tablet_replica.cc:330] T f574bc207ebd4605b1940daf4e8d7eb1 P f0a368c2bba649cfa232d7321568e655: stopping tablet replica
I20260811 11:55:58.906435 15511 raft_consensus.cc:2229] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:55:58.906468 15512 raft_consensus.cc:2229] T f574bc207ebd4605b1940daf4e8d7eb1 P f0a368c2bba649cfa232d7321568e655 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:55:58.906507 15222 tablet_service.cc:1514] Processing DeleteTablet for tablet f574bc207ebd4605b1940daf4e8d7eb1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:58 UTC) from {username='oryx', principal='oryx/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:50462
I20260811 11:55:58.906574 15512 raft_consensus.cc:2258] T f574bc207ebd4605b1940daf4e8d7eb1 P f0a368c2bba649cfa232d7321568e655 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:58.906669 15513 tablet_replica.cc:330] T f574bc207ebd4605b1940daf4e8d7eb1 P 87106c511d364f078ea5c1aa4a8be5dd: stopping tablet replica
I20260811 11:55:58.906765 15513 raft_consensus.cc:2229] T f574bc207ebd4605b1940daf4e8d7eb1 P 87106c511d364f078ea5c1aa4a8be5dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:55:58.906754 15511 raft_consensus.cc:2258] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:58.906862 15513 raft_consensus.cc:2258] T f574bc207ebd4605b1940daf4e8d7eb1 P 87106c511d364f078ea5c1aa4a8be5dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:58.907428 15512 ts_tablet_manager.cc:1905] T f574bc207ebd4605b1940daf4e8d7eb1 P f0a368c2bba649cfa232d7321568e655: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:58.907536 15511 ts_tablet_manager.cc:1905] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:58.907641 15513 ts_tablet_manager.cc:1905] T f574bc207ebd4605b1940daf4e8d7eb1 P 87106c511d364f078ea5c1aa4a8be5dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:58.910264 15511 ts_tablet_manager.cc:1918] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 11:55:58.910359 15511 log.cc:1196] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-2/wal/wals/f574bc207ebd4605b1940daf4e8d7eb1
I20260811 11:55:58.910684 15511 ts_tablet_manager.cc:1939] T f574bc207ebd4605b1940daf4e8d7eb1 P 0042fe01318b4642937a2a7e5fc7f97f: Deleting consensus metadata
I20260811 11:55:58.911428 14898 catalog_manager.cc:4841] TS 0042fe01318b4642937a2a7e5fc7f97f (127.6.143.131:46615): tablet f574bc207ebd4605b1940daf4e8d7eb1 (table test-table [id=5de9d9aabd9944289b74a2309a52725d]) successfully deleted
I20260811 11:55:58.911834 15512 ts_tablet_manager.cc:1918] T f574bc207ebd4605b1940daf4e8d7eb1 P f0a368c2bba649cfa232d7321568e655: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 11:55:58.912170 15512 log.cc:1196] T f574bc207ebd4605b1940daf4e8d7eb1 P f0a368c2bba649cfa232d7321568e655: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-0/wal/wals/f574bc207ebd4605b1940daf4e8d7eb1
I20260811 11:55:58.912443 15512 ts_tablet_manager.cc:1939] T f574bc207ebd4605b1940daf4e8d7eb1 P f0a368c2bba649cfa232d7321568e655: Deleting consensus metadata
I20260811 11:55:58.914412 14896 catalog_manager.cc:4841] TS f0a368c2bba649cfa232d7321568e655 (127.6.143.129:40905): tablet f574bc207ebd4605b1940daf4e8d7eb1 (table test-table [id=5de9d9aabd9944289b74a2309a52725d]) successfully deleted
I20260811 11:55:58.916083 15513 ts_tablet_manager.cc:1918] T f574bc207ebd4605b1940daf4e8d7eb1 P 87106c511d364f078ea5c1aa4a8be5dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 11:55:58.916216 15513 log.cc:1196] T f574bc207ebd4605b1940daf4e8d7eb1 P 87106c511d364f078ea5c1aa4a8be5dd: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-1/wal/wals/f574bc207ebd4605b1940daf4e8d7eb1
I20260811 11:55:58.916518 15513 ts_tablet_manager.cc:1939] T f574bc207ebd4605b1940daf4e8d7eb1 P 87106c511d364f078ea5c1aa4a8be5dd: Deleting consensus metadata
I20260811 11:55:58.917200 14896 catalog_manager.cc:4841] TS 87106c511d364f078ea5c1aa4a8be5dd (127.6.143.130:37463): tablet f574bc207ebd4605b1940daf4e8d7eb1 (table test-table [id=5de9d9aabd9944289b74a2309a52725d]) successfully deleted
Aug 11 11:55:58 dist-test-slave-zkpd krb5kdc[14860](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 11:55:58 dist-test-slave-zkpd krb5kdc[14860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:55:58 dist-test-slave-zkpd krb5kdc[14860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 11:55:58.952761 14966 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:55:58.944616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55206 (local address 127.6.143.190:39159)
0811 11:55:58.944764 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:58.944769 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:58.945155 (+ 386us) server_negotiation.cc:407] Connection header received
0811 11:55:58.945409 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:58.945412 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:58.945462 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:58.945517 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:58.946494 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.947354 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.948656 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.949000 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.949070 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.952636 (+ 3566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55206: BlockingRecv error: recv got EOF from 127.0.0.1:55206 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 11:55:58 dist-test-slave-zkpd krb5kdc[14860](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449358, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM
I20260811 11:55:58.966387 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:58.955263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55212 (local address 127.6.143.190:39159)
0811 11:55:58.955388 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:58.955392 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:58.955827 (+ 435us) server_negotiation.cc:407] Connection header received
0811 11:55:58.956040 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:58.956043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:58.956090 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:58.956141 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:55:58.956945 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.957707 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.958850 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:58.959191 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:58.959262 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:58.962429 (+ 3167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:55:58.962433 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:55:58.962436 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:55:58.962463 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:55:58.964521 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:58.965008 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:58.965012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:58.965014 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:58.965107 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:55:58.965496 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:55:58.965499 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:55:58.965501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:55:58.965721 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:55:58.965861 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:58.966102 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 11:55:58.966222 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 11:55:58.969291 14909 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:55:58.969800 14909 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:55:58.974834 15222 tablet_service.cc:1467] Processing CreateTablet for tablet 6b51c39fb9aa41eea77af9a03df11d30 (DEFAULT_TABLE table=test-table [id=5b94fbc1c31c4f56bd1bc4be46acc9d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:58.974957 15380 tablet_service.cc:1467] Processing CreateTablet for tablet 6b51c39fb9aa41eea77af9a03df11d30 (DEFAULT_TABLE table=test-table [id=5b94fbc1c31c4f56bd1bc4be46acc9d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:58.975126 15222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b51c39fb9aa41eea77af9a03df11d30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:58.975261 15380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b51c39fb9aa41eea77af9a03df11d30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:58.976370 15063 tablet_service.cc:1467] Processing CreateTablet for tablet 6b51c39fb9aa41eea77af9a03df11d30 (DEFAULT_TABLE table=test-table [id=5b94fbc1c31c4f56bd1bc4be46acc9d6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:58.976660 15063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6b51c39fb9aa41eea77af9a03df11d30. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:58.977509 15489 tablet_bootstrap.cc:492] T 6b51c39fb9aa41eea77af9a03df11d30 P 87106c511d364f078ea5c1aa4a8be5dd: Bootstrap starting.
I20260811 11:55:58.977799 15490 tablet_bootstrap.cc:492] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f: Bootstrap starting.
I20260811 11:55:58.978307 15489 tablet_bootstrap.cc:654] T 6b51c39fb9aa41eea77af9a03df11d30 P 87106c511d364f078ea5c1aa4a8be5dd: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:58.978773 15490 tablet_bootstrap.cc:654] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:58.979108 15488 tablet_bootstrap.cc:492] T 6b51c39fb9aa41eea77af9a03df11d30 P f0a368c2bba649cfa232d7321568e655: Bootstrap starting.
I20260811 11:55:58.979992 15488 tablet_bootstrap.cc:654] T 6b51c39fb9aa41eea77af9a03df11d30 P f0a368c2bba649cfa232d7321568e655: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:58.980226 15490 tablet_bootstrap.cc:492] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f: No bootstrap required, opened a new log
I20260811 11:55:58.980322 15490 ts_tablet_manager.cc:1397] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 11:55:58.980822 15490 raft_consensus.cc:348] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:58.980942 15490 raft_consensus.cc:374] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:58.980955 15489 tablet_bootstrap.cc:492] T 6b51c39fb9aa41eea77af9a03df11d30 P 87106c511d364f078ea5c1aa4a8be5dd: No bootstrap required, opened a new log
I20260811 11:55:58.980979 15490 raft_consensus.cc:729] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0042fe01318b4642937a2a7e5fc7f97f, State: Initialized, Role: FOLLOWER
I20260811 11:55:58.981029 15489 ts_tablet_manager.cc:1397] T 6b51c39fb9aa41eea77af9a03df11d30 P 87106c511d364f078ea5c1aa4a8be5dd: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 11:55:58.981101 15490 consensus_queue.cc:260] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:58.981148 15488 tablet_bootstrap.cc:492] T 6b51c39fb9aa41eea77af9a03df11d30 P f0a368c2bba649cfa232d7321568e655: No bootstrap required, opened a new log
I20260811 11:55:58.981218 15488 ts_tablet_manager.cc:1397] T 6b51c39fb9aa41eea77af9a03df11d30 P f0a368c2bba649cfa232d7321568e655: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260811 11:55:58.981367 15490 ts_tablet_manager.cc:1428] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:55:58.981418 15489 raft_consensus.cc:348] T 6b51c39fb9aa41eea77af9a03df11d30 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:58.981520 15489 raft_consensus.cc:374] T 6b51c39fb9aa41eea77af9a03df11d30 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:58.981577 15489 raft_consensus.cc:729] T 6b51c39fb9aa41eea77af9a03df11d30 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87106c511d364f078ea5c1aa4a8be5dd, State: Initialized, Role: FOLLOWER
I20260811 11:55:58.981603 15488 raft_consensus.cc:348] T 6b51c39fb9aa41eea77af9a03df11d30 P f0a368c2bba649cfa232d7321568e655 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:58.981702 15488 raft_consensus.cc:374] T 6b51c39fb9aa41eea77af9a03df11d30 P f0a368c2bba649cfa232d7321568e655 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:58.981734 15488 raft_consensus.cc:729] T 6b51c39fb9aa41eea77af9a03df11d30 P f0a368c2bba649cfa232d7321568e655 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0a368c2bba649cfa232d7321568e655, State: Initialized, Role: FOLLOWER
I20260811 11:55:58.981690 15489 consensus_queue.cc:260] T 6b51c39fb9aa41eea77af9a03df11d30 P 87106c511d364f078ea5c1aa4a8be5dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:58.981842 15488 consensus_queue.cc:260] T 6b51c39fb9aa41eea77af9a03df11d30 P f0a368c2bba649cfa232d7321568e655 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:58.981969 15489 ts_tablet_manager.cc:1428] T 6b51c39fb9aa41eea77af9a03df11d30 P 87106c511d364f078ea5c1aa4a8be5dd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:55:58.982115 15488 ts_tablet_manager.cc:1428] T 6b51c39fb9aa41eea77af9a03df11d30 P f0a368c2bba649cfa232d7321568e655: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 11:55:59.008029 15494 raft_consensus.cc:482] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:55:59.008160 15494 raft_consensus.cc:504] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:59.008591 15494 leader_election.cc:288] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0a368c2bba649cfa232d7321568e655 (127.6.143.129:40905), 87106c511d364f078ea5c1aa4a8be5dd (127.6.143.130:37463)
I20260811 11:55:59.009029 15085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b51c39fb9aa41eea77af9a03df11d30" candidate_uuid: "0042fe01318b4642937a2a7e5fc7f97f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0a368c2bba649cfa232d7321568e655" is_pre_election: true
I20260811 11:55:59.009052 15244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b51c39fb9aa41eea77af9a03df11d30" candidate_uuid: "0042fe01318b4642937a2a7e5fc7f97f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87106c511d364f078ea5c1aa4a8be5dd" is_pre_election: true
I20260811 11:55:59.009181 15085 raft_consensus.cc:2454] T 6b51c39fb9aa41eea77af9a03df11d30 P f0a368c2bba649cfa232d7321568e655 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0042fe01318b4642937a2a7e5fc7f97f in term 0.
I20260811 11:55:59.009189 15244 raft_consensus.cc:2454] T 6b51c39fb9aa41eea77af9a03df11d30 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0042fe01318b4642937a2a7e5fc7f97f in term 0.
I20260811 11:55:59.009539 15326 leader_election.cc:302] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0042fe01318b4642937a2a7e5fc7f97f, 87106c511d364f078ea5c1aa4a8be5dd; no voters:
I20260811 11:55:59.009758 15494 raft_consensus.cc:2781] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:55:59.009827 15494 raft_consensus.cc:482] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:55:59.009874 15494 raft_consensus.cc:3037] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:59.010998 15494 raft_consensus.cc:504] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:59.011318 15494 leader_election.cc:288] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f [CANDIDATE]: Term 1 election: Requested vote from peers f0a368c2bba649cfa232d7321568e655 (127.6.143.129:40905), 87106c511d364f078ea5c1aa4a8be5dd (127.6.143.130:37463)
I20260811 11:55:59.011667 15085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b51c39fb9aa41eea77af9a03df11d30" candidate_uuid: "0042fe01318b4642937a2a7e5fc7f97f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0a368c2bba649cfa232d7321568e655"
I20260811 11:55:59.011737 15244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6b51c39fb9aa41eea77af9a03df11d30" candidate_uuid: "0042fe01318b4642937a2a7e5fc7f97f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87106c511d364f078ea5c1aa4a8be5dd"
I20260811 11:55:59.011804 15085 raft_consensus.cc:3037] T 6b51c39fb9aa41eea77af9a03df11d30 P f0a368c2bba649cfa232d7321568e655 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:59.011894 15244 raft_consensus.cc:3037] T 6b51c39fb9aa41eea77af9a03df11d30 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:59.013095 15244 raft_consensus.cc:2454] T 6b51c39fb9aa41eea77af9a03df11d30 P 87106c511d364f078ea5c1aa4a8be5dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0042fe01318b4642937a2a7e5fc7f97f in term 1.
I20260811 11:55:59.013140 15085 raft_consensus.cc:2454] T 6b51c39fb9aa41eea77af9a03df11d30 P f0a368c2bba649cfa232d7321568e655 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0042fe01318b4642937a2a7e5fc7f97f in term 1.
I20260811 11:55:59.013409 15326 leader_election.cc:302] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0042fe01318b4642937a2a7e5fc7f97f, 87106c511d364f078ea5c1aa4a8be5dd; no voters:
I20260811 11:55:59.013620 15494 raft_consensus.cc:2781] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:59.013742 15494 raft_consensus.cc:686] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f [term 1 LEADER]: Becoming Leader. State: Replica: 0042fe01318b4642937a2a7e5fc7f97f, State: Running, Role: LEADER
I20260811 11:55:59.013876 15494 consensus_queue.cc:237] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:59.015586 14910 catalog_manager.cc:5495] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0042fe01318b4642937a2a7e5fc7f97f (127.6.143.131). New cstate: current_term: 1 leader_uuid: "0042fe01318b4642937a2a7e5fc7f97f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } health_report { overall_health: HEALTHY } } }
I20260811 11:55:59.025746 15483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:59.021947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50724 (local address 127.6.143.131:46615)
0811 11:55:59.022080 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:59.022085 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:59.022098 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:55:59.022211 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:59.022216 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:59.022273 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:59.022379 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:55:59.022645 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:59.023453 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:59.024423 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:59.024799 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:59.024943 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:59.025298 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:55:59.025386 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:55:59.025464 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:59.025549 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 11:55:59.025604 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 11:55:59.027684 15085 raft_consensus.cc:1261] T 6b51c39fb9aa41eea77af9a03df11d30 P f0a368c2bba649cfa232d7321568e655 [term 1 FOLLOWER]: Refusing update from remote peer 0042fe01318b4642937a2a7e5fc7f97f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:59.027719 15244 raft_consensus.cc:1261] T 6b51c39fb9aa41eea77af9a03df11d30 P 87106c511d364f078ea5c1aa4a8be5dd [term 1 FOLLOWER]: Refusing update from remote peer 0042fe01318b4642937a2a7e5fc7f97f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:59.028162 15549 consensus_queue.cc:1035] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f [LEADER]: Connected to new peer: Peer: permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:59.028311 15494 consensus_queue.cc:1035] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:59.037674 14911 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:55:59.037828 14911 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:55:59.039736 14911 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f9ce963fd5964673929ebfb3cfbf4f27: Sending DeleteTablet for 3 replicas of tablet 6b51c39fb9aa41eea77af9a03df11d30
I20260811 11:55:59.040331 15063 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b51c39fb9aa41eea77af9a03df11d30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:59 UTC) from {username='oryx', principal='oryx/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:39422
I20260811 11:55:59.040324 15222 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b51c39fb9aa41eea77af9a03df11d30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:59 UTC) from {username='oryx', principal='oryx/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:50462
I20260811 11:55:59.040517 15512 tablet_replica.cc:330] T 6b51c39fb9aa41eea77af9a03df11d30 P f0a368c2bba649cfa232d7321568e655: stopping tablet replica
I20260811 11:55:59.040524 15513 tablet_replica.cc:330] T 6b51c39fb9aa41eea77af9a03df11d30 P 87106c511d364f078ea5c1aa4a8be5dd: stopping tablet replica
I20260811 11:55:59.040633 15512 raft_consensus.cc:2229] T 6b51c39fb9aa41eea77af9a03df11d30 P f0a368c2bba649cfa232d7321568e655 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:55:59.040642 15513 raft_consensus.cc:2229] T 6b51c39fb9aa41eea77af9a03df11d30 P 87106c511d364f078ea5c1aa4a8be5dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:55:59.040778 15512 raft_consensus.cc:2258] T 6b51c39fb9aa41eea77af9a03df11d30 P f0a368c2bba649cfa232d7321568e655 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:59.040784 15513 raft_consensus.cc:2258] T 6b51c39fb9aa41eea77af9a03df11d30 P 87106c511d364f078ea5c1aa4a8be5dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:59.041195 15380 tablet_service.cc:1514] Processing DeleteTablet for tablet 6b51c39fb9aa41eea77af9a03df11d30 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:59 UTC) from {username='oryx', principal='oryx/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:50706
I20260811 11:55:59.041352 15511 tablet_replica.cc:330] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f: stopping tablet replica
I20260811 11:55:59.041468 15511 raft_consensus.cc:2229] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:55:59.041648 15511 raft_consensus.cc:2258] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:59.041893 14911 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 11:55:59.042266 15511 ts_tablet_manager.cc:1905] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:59.042289 15513 ts_tablet_manager.cc:1905] T 6b51c39fb9aa41eea77af9a03df11d30 P 87106c511d364f078ea5c1aa4a8be5dd: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 11:55:59.042486 14911 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:55:59.044160 15512 ts_tablet_manager.cc:1905] T 6b51c39fb9aa41eea77af9a03df11d30 P f0a368c2bba649cfa232d7321568e655: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:59.044822 15513 ts_tablet_manager.cc:1918] T 6b51c39fb9aa41eea77af9a03df11d30 P 87106c511d364f078ea5c1aa4a8be5dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:55:59.044898 15513 log.cc:1196] T 6b51c39fb9aa41eea77af9a03df11d30 P 87106c511d364f078ea5c1aa4a8be5dd: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-1/wal/wals/6b51c39fb9aa41eea77af9a03df11d30
I20260811 11:55:59.045148 15513 ts_tablet_manager.cc:1939] T 6b51c39fb9aa41eea77af9a03df11d30 P 87106c511d364f078ea5c1aa4a8be5dd: Deleting consensus metadata
I20260811 11:55:59.045809 14896 catalog_manager.cc:4841] TS 87106c511d364f078ea5c1aa4a8be5dd (127.6.143.130:37463): tablet 6b51c39fb9aa41eea77af9a03df11d30 (table test-table [id=5b94fbc1c31c4f56bd1bc4be46acc9d6]) successfully deleted
I20260811 11:55:59.045940 15511 ts_tablet_manager.cc:1918] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:55:59.046025 15511 log.cc:1196] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-2/wal/wals/6b51c39fb9aa41eea77af9a03df11d30
I20260811 11:55:59.046268 15511 ts_tablet_manager.cc:1939] T 6b51c39fb9aa41eea77af9a03df11d30 P 0042fe01318b4642937a2a7e5fc7f97f: Deleting consensus metadata
I20260811 11:55:59.047065 14898 catalog_manager.cc:4841] TS 0042fe01318b4642937a2a7e5fc7f97f (127.6.143.131:46615): tablet 6b51c39fb9aa41eea77af9a03df11d30 (table test-table [id=5b94fbc1c31c4f56bd1bc4be46acc9d6]) successfully deleted
I20260811 11:55:59.047467 15512 ts_tablet_manager.cc:1918] T 6b51c39fb9aa41eea77af9a03df11d30 P f0a368c2bba649cfa232d7321568e655: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:55:59.047518 15512 log.cc:1196] T 6b51c39fb9aa41eea77af9a03df11d30 P f0a368c2bba649cfa232d7321568e655: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-0/wal/wals/6b51c39fb9aa41eea77af9a03df11d30
I20260811 11:55:59.047694 15512 ts_tablet_manager.cc:1939] T 6b51c39fb9aa41eea77af9a03df11d30 P f0a368c2bba649cfa232d7321568e655: Deleting consensus metadata
I20260811 11:55:59.048362 14896 catalog_manager.cc:4841] TS f0a368c2bba649cfa232d7321568e655 (127.6.143.129:40905): tablet 6b51c39fb9aa41eea77af9a03df11d30 (table test-table [id=5b94fbc1c31c4f56bd1bc4be46acc9d6]) successfully deleted
I20260811 11:55:59.049695 15063 tablet_service.cc:1467] Processing CreateTablet for tablet b8d83d9f6541489f8c3dc83097f80129 (DEFAULT_TABLE table=test-table [id=e32e54c4b4a649eb9c5785cc5cc4991f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:59.049809 15222 tablet_service.cc:1467] Processing CreateTablet for tablet b8d83d9f6541489f8c3dc83097f80129 (DEFAULT_TABLE table=test-table [id=e32e54c4b4a649eb9c5785cc5cc4991f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:59.049979 15063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8d83d9f6541489f8c3dc83097f80129. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:59.050077 15222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8d83d9f6541489f8c3dc83097f80129. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:59.051605 15488 tablet_bootstrap.cc:492] T b8d83d9f6541489f8c3dc83097f80129 P f0a368c2bba649cfa232d7321568e655: Bootstrap starting.
I20260811 11:55:59.052480 15380 tablet_service.cc:1467] Processing CreateTablet for tablet b8d83d9f6541489f8c3dc83097f80129 (DEFAULT_TABLE table=test-table [id=e32e54c4b4a649eb9c5785cc5cc4991f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:55:59.052610 15488 tablet_bootstrap.cc:654] T b8d83d9f6541489f8c3dc83097f80129 P f0a368c2bba649cfa232d7321568e655: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:59.052714 15380 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b8d83d9f6541489f8c3dc83097f80129. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:55:59.053210 15489 tablet_bootstrap.cc:492] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd: Bootstrap starting.
I20260811 11:55:59.053892 15489 tablet_bootstrap.cc:654] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:59.054886 15489 tablet_bootstrap.cc:492] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd: No bootstrap required, opened a new log
I20260811 11:55:59.054968 15490 tablet_bootstrap.cc:492] T b8d83d9f6541489f8c3dc83097f80129 P 0042fe01318b4642937a2a7e5fc7f97f: Bootstrap starting.
I20260811 11:55:59.054967 15489 ts_tablet_manager.cc:1397] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 11:55:59.055334 15489 raft_consensus.cc:348] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:59.055441 15489 raft_consensus.cc:374] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:59.055477 15489 raft_consensus.cc:729] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87106c511d364f078ea5c1aa4a8be5dd, State: Initialized, Role: FOLLOWER
I20260811 11:55:59.055598 15489 consensus_queue.cc:260] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:59.055848 15490 tablet_bootstrap.cc:654] T b8d83d9f6541489f8c3dc83097f80129 P 0042fe01318b4642937a2a7e5fc7f97f: Neither blocks nor log segments found. Creating new log.
I20260811 11:55:59.055939 15489 ts_tablet_manager.cc:1428] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:55:59.056731 15488 tablet_bootstrap.cc:492] T b8d83d9f6541489f8c3dc83097f80129 P f0a368c2bba649cfa232d7321568e655: No bootstrap required, opened a new log
I20260811 11:55:59.056816 15488 ts_tablet_manager.cc:1397] T b8d83d9f6541489f8c3dc83097f80129 P f0a368c2bba649cfa232d7321568e655: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s
I20260811 11:55:59.057175 15490 tablet_bootstrap.cc:492] T b8d83d9f6541489f8c3dc83097f80129 P 0042fe01318b4642937a2a7e5fc7f97f: No bootstrap required, opened a new log
I20260811 11:55:59.057253 15490 ts_tablet_manager.cc:1397] T b8d83d9f6541489f8c3dc83097f80129 P 0042fe01318b4642937a2a7e5fc7f97f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:55:59.057235 15488 raft_consensus.cc:348] T b8d83d9f6541489f8c3dc83097f80129 P f0a368c2bba649cfa232d7321568e655 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:59.057375 15488 raft_consensus.cc:374] T b8d83d9f6541489f8c3dc83097f80129 P f0a368c2bba649cfa232d7321568e655 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:59.057415 15488 raft_consensus.cc:729] T b8d83d9f6541489f8c3dc83097f80129 P f0a368c2bba649cfa232d7321568e655 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0a368c2bba649cfa232d7321568e655, State: Initialized, Role: FOLLOWER
I20260811 11:55:59.057540 15488 consensus_queue.cc:260] T b8d83d9f6541489f8c3dc83097f80129 P f0a368c2bba649cfa232d7321568e655 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:59.057708 15490 raft_consensus.cc:348] T b8d83d9f6541489f8c3dc83097f80129 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:59.057811 15490 raft_consensus.cc:374] T b8d83d9f6541489f8c3dc83097f80129 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:55:59.057818 15488 ts_tablet_manager.cc:1428] T b8d83d9f6541489f8c3dc83097f80129 P f0a368c2bba649cfa232d7321568e655: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 11:55:59.057844 15490 raft_consensus.cc:729] T b8d83d9f6541489f8c3dc83097f80129 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0042fe01318b4642937a2a7e5fc7f97f, State: Initialized, Role: FOLLOWER
I20260811 11:55:59.057967 15490 consensus_queue.cc:260] T b8d83d9f6541489f8c3dc83097f80129 P 0042fe01318b4642937a2a7e5fc7f97f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:59.058243 15490 ts_tablet_manager.cc:1428] T b8d83d9f6541489f8c3dc83097f80129 P 0042fe01318b4642937a2a7e5fc7f97f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:55:59.113106 15499 raft_consensus.cc:482] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:55:59.113248 15499 raft_consensus.cc:504] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:59.113768 15499 leader_election.cc:288] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0a368c2bba649cfa232d7321568e655 (127.6.143.129:40905), 0042fe01318b4642937a2a7e5fc7f97f (127.6.143.131:46615)
I20260811 11:55:59.114173 15085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8d83d9f6541489f8c3dc83097f80129" candidate_uuid: "87106c511d364f078ea5c1aa4a8be5dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0a368c2bba649cfa232d7321568e655" is_pre_election: true
I20260811 11:55:59.114203 15401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8d83d9f6541489f8c3dc83097f80129" candidate_uuid: "87106c511d364f078ea5c1aa4a8be5dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0042fe01318b4642937a2a7e5fc7f97f" is_pre_election: true
I20260811 11:55:59.114342 15401 raft_consensus.cc:2454] T b8d83d9f6541489f8c3dc83097f80129 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87106c511d364f078ea5c1aa4a8be5dd in term 0.
I20260811 11:55:59.114343 15085 raft_consensus.cc:2454] T b8d83d9f6541489f8c3dc83097f80129 P f0a368c2bba649cfa232d7321568e655 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87106c511d364f078ea5c1aa4a8be5dd in term 0.
I20260811 11:55:59.114693 15170 leader_election.cc:302] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0042fe01318b4642937a2a7e5fc7f97f, 87106c511d364f078ea5c1aa4a8be5dd, f0a368c2bba649cfa232d7321568e655; no voters:
I20260811 11:55:59.114943 15499 raft_consensus.cc:2781] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:55:59.115032 15499 raft_consensus.cc:482] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:55:59.115073 15499 raft_consensus.cc:3037] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:59.116343 15499 raft_consensus.cc:504] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:59.116679 15499 leader_election.cc:288] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd [CANDIDATE]: Term 1 election: Requested vote from peers f0a368c2bba649cfa232d7321568e655 (127.6.143.129:40905), 0042fe01318b4642937a2a7e5fc7f97f (127.6.143.131:46615)
I20260811 11:55:59.117004 15085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8d83d9f6541489f8c3dc83097f80129" candidate_uuid: "87106c511d364f078ea5c1aa4a8be5dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0a368c2bba649cfa232d7321568e655"
I20260811 11:55:59.117137 15085 raft_consensus.cc:3037] T b8d83d9f6541489f8c3dc83097f80129 P f0a368c2bba649cfa232d7321568e655 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:59.117103 15401 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b8d83d9f6541489f8c3dc83097f80129" candidate_uuid: "87106c511d364f078ea5c1aa4a8be5dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0042fe01318b4642937a2a7e5fc7f97f"
I20260811 11:55:59.117220 15401 raft_consensus.cc:3037] T b8d83d9f6541489f8c3dc83097f80129 P 0042fe01318b4642937a2a7e5fc7f97f [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:55:59.118295 15401 raft_consensus.cc:2454] T b8d83d9f6541489f8c3dc83097f80129 P 0042fe01318b4642937a2a7e5fc7f97f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87106c511d364f078ea5c1aa4a8be5dd in term 1.
I20260811 11:55:59.118295 15085 raft_consensus.cc:2454] T b8d83d9f6541489f8c3dc83097f80129 P f0a368c2bba649cfa232d7321568e655 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87106c511d364f078ea5c1aa4a8be5dd in term 1.
I20260811 11:55:59.118587 15170 leader_election.cc:302] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0042fe01318b4642937a2a7e5fc7f97f, 87106c511d364f078ea5c1aa4a8be5dd; no voters:
I20260811 11:55:59.118779 15499 raft_consensus.cc:2781] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:55:59.118912 15499 raft_consensus.cc:686] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd [term 1 LEADER]: Becoming Leader. State: Replica: 87106c511d364f078ea5c1aa4a8be5dd, State: Running, Role: LEADER
I20260811 11:55:59.119092 15499 consensus_queue.cc:237] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } }
I20260811 11:55:59.120642 14910 catalog_manager.cc:5495] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 87106c511d364f078ea5c1aa4a8be5dd (127.6.143.130). New cstate: current_term: 1 leader_uuid: "87106c511d364f078ea5c1aa4a8be5dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87106c511d364f078ea5c1aa4a8be5dd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37463 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 } health_report { overall_health: UNKNOWN } } }
I20260811 11:55:59.149277 15487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:59.146396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50498 (local address 127.6.143.130:37463)
0811 11:55:59.146529 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:59.146533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:59.146585 (+ 52us) server_negotiation.cc:407] Connection header received
0811 11:55:59.146709 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:59.146712 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:59.146753 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:59.146808 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:55:59.147015 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:59.147500 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:59.148364 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:59.148576 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:59.148658 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:59.148921 (+ 263us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:55:59.148990 (+ 69us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:55:59.149045 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:59.149130 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 11:55:59.149170 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 11:55:59.157150 15487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:59.154636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:45153 (local address 127.6.143.130:37463)
0811 11:55:59.154748 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 11:55:59.154751 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:55:59.154762 (+ 11us) server_negotiation.cc:407] Connection header received
0811 11:55:59.154840 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:55:59.154842 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:55:59.154879 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:55:59.154944 (+ 65us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:55:59.155112 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:59.155558 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:59.156317 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:55:59.156519 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:55:59.156596 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:59.156791 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:55:59.156847 (+ 56us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:55:59.156906 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 11:55:59.156977 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 11:55:59.157031 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":46}
I20260811 11:55:59.157171 15551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:59.154574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:37463 (local address 127.6.143.130:45153)
0811 11:55:59.154701 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:59.154713 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:55:59.154778 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:55:59.154969 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:55:59.154971 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:55:59.154977 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 11:55:59.155049 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:59.155053 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:59.155642 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:59.155644 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:59.156222 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:55:59.156230 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:55:59.156614 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:55:59.156617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:55:59.156673 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:55:59.156698 (+ 25us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 11:55:59.156918 (+ 220us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 11:55:59.156922 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:55:59.156972 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 11:55:59.157062 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":15}
I20260811 11:55:59.162828 15085 raft_consensus.cc:1261] T b8d83d9f6541489f8c3dc83097f80129 P f0a368c2bba649cfa232d7321568e655 [term 1 FOLLOWER]: Refusing update from remote peer 87106c511d364f078ea5c1aa4a8be5dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:59.162901 15401 raft_consensus.cc:1261] T b8d83d9f6541489f8c3dc83097f80129 P 0042fe01318b4642937a2a7e5fc7f97f [term 1 FOLLOWER]: Refusing update from remote peer 87106c511d364f078ea5c1aa4a8be5dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:55:59.163298 15499 consensus_queue.cc:1035] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0a368c2bba649cfa232d7321568e655" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 40905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:59.163501 15546 consensus_queue.cc:1035] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "0042fe01318b4642937a2a7e5fc7f97f" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 46615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:55:59.200802 14910 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:55:59.200959 14910 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:55:59.202775 14910 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f9ce963fd5964673929ebfb3cfbf4f27: Sending DeleteTablet for 3 replicas of tablet b8d83d9f6541489f8c3dc83097f80129
I20260811 11:55:59.203315 15063 tablet_service.cc:1514] Processing DeleteTablet for tablet b8d83d9f6541489f8c3dc83097f80129 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:59 UTC) from {username='oryx', principal='oryx/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:39422
I20260811 11:55:59.203351 15222 tablet_service.cc:1514] Processing DeleteTablet for tablet b8d83d9f6541489f8c3dc83097f80129 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:59 UTC) from {username='oryx', principal='oryx/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:50462
I20260811 11:55:59.203441 15380 tablet_service.cc:1514] Processing DeleteTablet for tablet b8d83d9f6541489f8c3dc83097f80129 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:55:59 UTC) from {username='oryx', principal='oryx/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:50706
I20260811 11:55:59.203503 15513 tablet_replica.cc:330] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd: stopping tablet replica
I20260811 11:55:59.203504 15512 tablet_replica.cc:330] T b8d83d9f6541489f8c3dc83097f80129 P f0a368c2bba649cfa232d7321568e655: stopping tablet replica
I20260811 11:55:59.203611 15511 tablet_replica.cc:330] T b8d83d9f6541489f8c3dc83097f80129 P 0042fe01318b4642937a2a7e5fc7f97f: stopping tablet replica
I20260811 11:55:59.203617 15513 raft_consensus.cc:2229] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:55:59.203676 15512 raft_consensus.cc:2229] T b8d83d9f6541489f8c3dc83097f80129 P f0a368c2bba649cfa232d7321568e655 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:55:59.203709 15511 raft_consensus.cc:2229] T b8d83d9f6541489f8c3dc83097f80129 P 0042fe01318b4642937a2a7e5fc7f97f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:55:59.203811 15511 raft_consensus.cc:2258] T b8d83d9f6541489f8c3dc83097f80129 P 0042fe01318b4642937a2a7e5fc7f97f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:59.203809 15513 raft_consensus.cc:2258] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:59.203850 15512 raft_consensus.cc:2258] T b8d83d9f6541489f8c3dc83097f80129 P f0a368c2bba649cfa232d7321568e655 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:55:59.204103 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 14991
I20260811 11:55:59.204700 15511 ts_tablet_manager.cc:1905] T b8d83d9f6541489f8c3dc83097f80129 P 0042fe01318b4642937a2a7e5fc7f97f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:59.206636 15513 ts_tablet_manager.cc:1905] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:55:59.208112 15511 ts_tablet_manager.cc:1918] T b8d83d9f6541489f8c3dc83097f80129 P 0042fe01318b4642937a2a7e5fc7f97f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 11:55:59.208215 15511 log.cc:1196] T b8d83d9f6541489f8c3dc83097f80129 P 0042fe01318b4642937a2a7e5fc7f97f: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-2/wal/wals/b8d83d9f6541489f8c3dc83097f80129
I20260811 11:55:59.208531 15511 ts_tablet_manager.cc:1939] T b8d83d9f6541489f8c3dc83097f80129 P 0042fe01318b4642937a2a7e5fc7f97f: Deleting consensus metadata
I20260811 11:55:59.208936 15513 ts_tablet_manager.cc:1918] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 11:55:59.209030 15513 log.cc:1196] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786449319379472-6718-0/minicluster-data/ts-1/wal/wals/b8d83d9f6541489f8c3dc83097f80129
I20260811 11:55:59.209204 14898 catalog_manager.cc:4841] TS 0042fe01318b4642937a2a7e5fc7f97f (127.6.143.131:46615): tablet b8d83d9f6541489f8c3dc83097f80129 (table test-table [id=e32e54c4b4a649eb9c5785cc5cc4991f]) successfully deleted
I20260811 11:55:59.209299 15513 ts_tablet_manager.cc:1939] T b8d83d9f6541489f8c3dc83097f80129 P 87106c511d364f078ea5c1aa4a8be5dd: Deleting consensus metadata
I20260811 11:55:59.210129 14896 catalog_manager.cc:4841] TS 87106c511d364f078ea5c1aa4a8be5dd (127.6.143.130:37463): tablet b8d83d9f6541489f8c3dc83097f80129 (table test-table [id=e32e54c4b4a649eb9c5785cc5cc4991f]) successfully deleted
W20260811 11:55:59.215166 14896 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.143.129:40905 (error 108)
I20260811 11:55:59.216044 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 15147
I20260811 11:55:59.216360 15585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:55:59.215851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:40905 (local address 127.0.0.1:39430)
0811 11:55:59.216145 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 11:55:59.216235 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.129:40905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":100,"threads_started":1}
W20260811 11:55:59.218580 14896 catalog_manager.cc:4584] TS f0a368c2bba649cfa232d7321568e655 (127.6.143.129:40905): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b8d83d9f6541489f8c3dc83097f80129: Network error: Client connection negotiation failed: client connection to 127.6.143.129:40905: connect: Connection refused (error 111)
I20260811 11:55:59.226459 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 15307
I20260811 11:55:59.234457 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 14876
2026-08-11T11:55:59Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3235 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:55:59 dist-test-slave-zkpd krb5kdc[15588](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:55:59 dist-test-slave-zkpd krb5kdc[15588](info): set up 2 sockets
Aug 11 11:55:59 dist-test-slave-zkpd krb5kdc[15588](info): commencing operation
krb5kdc: starting...
W20260811 11:56:00.333833 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:56:00 dist-test-slave-zkpd krb5kdc[15588](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449360, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:56:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:56:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
I20260811 11:56:00.456534 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:45725
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:42239
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:45725
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:56:00.570900 15602 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:56:00.571139 15602 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:56:00.571177 15602 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:56:00.575172 15602 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:56:00.575232 15602 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:56:00.575250 15602 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:56:00.575266 15602 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:56:00.575321 15602 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:56:00.580498 15602 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:42239
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:45725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:45725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15602
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:56:00.581550 15602 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:56:00.582595 15602 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:56:00.591522 15608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:00.591558 15610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:56:00.591675 15602 server_base.cc:1034] running on GCE node
W20260811 11:56:00.591619 15607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:56:00.592413 15602 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:56:00.593544 15602 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:56:00.594738 15602 hybrid_clock.cc:648] HybridClock initialized: now 1786449360594718 us; error 31 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T11:56:00Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1368 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:56:00 dist-test-slave-zkpd krb5kdc[15615](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:56:00 dist-test-slave-zkpd krb5kdc[15615](info): set up 2 sockets
Aug 11 11:56:00 dist-test-slave-zkpd krb5kdc[15615](info): commencing operation
krb5kdc: starting...
W20260811 11:56:01.690261 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:56:01 dist-test-slave-zkpd krb5kdc[15615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:56:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:56:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
I20260811 11:56:01.810885 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:46709
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:34565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:46709
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:56:01.921712 15631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:56:01.921931 15631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:56:01.921988 15631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:56:01.925885 15631 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:56:01.925933 15631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:56:01.925952 15631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:56:01.925989 15631 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:56:01.926023 15631 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:56:01.931041 15631 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:34565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:46709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:46709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:56:01.932088 15631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:56:01.933058 15631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:56:01.941044 15640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:01.941056 15642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:01.941056 15637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:56:01.941257 15631 server_base.cc:1034] running on GCE node
I20260811 11:56:01.941850 15631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:56:01.942911 15631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:56:01.944069 15631 hybrid_clock.cc:648] HybridClock initialized: now 1786449361944050 us; error 29 us; skew 500 ppm
Aug 11 11:56:01 dist-test-slave-zkpd krb5kdc[15615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449361, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:56:01.946727 15631 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:56:01.947515 15631 webserver.cc:460] Webserver started at http://127.6.143.190:45549/ using document root <none> and password file <none>
I20260811 11:56:01.948127 15631 fs_manager.cc:359] Metadata directory not provided
I20260811 11:56:01.948217 15631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:56:01.948446 15631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:56:01.950300 15631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "824caff9b6a44cc0ba1acd316ec815ee"
format_stamp: "Formatted at 2026-08-11 11:56:01 on dist-test-slave-zkpd"
server_key: "4241a3fa61c742e56b8b079a9141fb21"
server_key_iv: "a6f7dd12e1fad1f5047fadc1a7ef584f"
server_key_version: "encryptionkey@0"
I20260811 11:56:01.950877 15631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "824caff9b6a44cc0ba1acd316ec815ee"
format_stamp: "Formatted at 2026-08-11 11:56:01 on dist-test-slave-zkpd"
server_key: "4241a3fa61c742e56b8b079a9141fb21"
server_key_iv: "a6f7dd12e1fad1f5047fadc1a7ef584f"
server_key_version: "encryptionkey@0"
I20260811 11:56:01.954478 15631 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 11:56:01.956844 15650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:01.957957 15631 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:56:01.958112 15631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "824caff9b6a44cc0ba1acd316ec815ee"
format_stamp: "Formatted at 2026-08-11 11:56:01 on dist-test-slave-zkpd"
server_key: "4241a3fa61c742e56b8b079a9141fb21"
server_key_iv: "a6f7dd12e1fad1f5047fadc1a7ef584f"
server_key_version: "encryptionkey@0"
I20260811 11:56:01.958218 15631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:56:01.979878 15631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:56:01.982602 15631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:56:01.982827 15631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:56:01.989935 15631 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:46709
I20260811 11:56:01.989964 15711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:46709 every 8 connection(s)
I20260811 11:56:01.990849 15631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:56:01.993671 15712 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:56:01.996656 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 15631
I20260811 11:56:01.996757 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:56:01.997058 6718 external_mini_cluster.cc:1428] Setting key 686b89d04bed68cf41a12db0bb6bd10b
I20260811 11:56:01.999583 15712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee: Bootstrap starting.
I20260811 11:56:02.001793 15712 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee: Neither blocks nor log segments found. Creating new log.
I20260811 11:56:02.002539 15712 log.cc:824] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee: Log is configured to *not* fsync() on all Append() calls
I20260811 11:56:02.004703 15712 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee: No bootstrap required, opened a new log
Aug 11 11:56:02 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:56:02.009142 15712 raft_consensus.cc:348] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824caff9b6a44cc0ba1acd316ec815ee" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46709 } }
I20260811 11:56:02.009469 15712 raft_consensus.cc:374] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:56:02.009565 15712 raft_consensus.cc:729] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 824caff9b6a44cc0ba1acd316ec815ee, State: Initialized, Role: FOLLOWER
I20260811 11:56:02.010098 15712 consensus_queue.cc:260] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824caff9b6a44cc0ba1acd316ec815ee" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46709 } }
I20260811 11:56:02.010277 15712 raft_consensus.cc:388] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:56:02.010396 15712 raft_consensus.cc:482] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:56:02.010516 15712 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:56:02.011411 15712 raft_consensus.cc:504] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824caff9b6a44cc0ba1acd316ec815ee" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46709 } }
I20260811 11:56:02.011523 15715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:01.998448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52088 (local address 127.6.143.190:46709)
0811 11:56:01.998894 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:01.998909 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:01.998943 (+ 34us) server_negotiation.cc:407] Connection header received
0811 11:56:01.999551 (+ 608us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:01.999567 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:01.999996 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:02.000347 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:02.001166 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:02.002207 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:02.003121 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:02.003591 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:02.003771 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:02.006625 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:56:02.006630 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:56:02.006643 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:56:02.006669 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:56:02.008489 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:56:02.008994 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:56:02.008998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:56:02.009003 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:56:02.009112 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:56:02.009423 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:56:02.009425 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:56:02.009427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:56:02.009932 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:56:02.010088 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 11:56:02.010446 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 11:56:02.010778 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":132,"threads_started":1}
I20260811 11:56:02.011782 15712 leader_election.cc:302] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 824caff9b6a44cc0ba1acd316ec815ee; no voters:
I20260811 11:56:02.012145 15712 leader_election.cc:288] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:56:02.012236 15718 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:56:02.012425 15718 raft_consensus.cc:686] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [term 1 LEADER]: Becoming Leader. State: Replica: 824caff9b6a44cc0ba1acd316ec815ee, State: Running, Role: LEADER
I20260811 11:56:02.012796 15718 consensus_queue.cc:237] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824caff9b6a44cc0ba1acd316ec815ee" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46709 } }
I20260811 11:56:02.013370 15712 sys_catalog.cc:564] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:56:02.013978 15720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 824caff9b6a44cc0ba1acd316ec815ee. Latest consensus state: current_term: 1 leader_uuid: "824caff9b6a44cc0ba1acd316ec815ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824caff9b6a44cc0ba1acd316ec815ee" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46709 } } }
I20260811 11:56:02.014199 15720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [sys.catalog]: This master's current role is: LEADER
I20260811 11:56:02.014438 15719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "824caff9b6a44cc0ba1acd316ec815ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824caff9b6a44cc0ba1acd316ec815ee" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46709 } } }
I20260811 11:56:02.014590 15719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [sys.catalog]: This master's current role is: LEADER
I20260811 11:56:02.014711 15726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:56:02.017612 15726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:56:02.022660 15726 catalog_manager.cc:1347] Generated new cluster ID: ba84c562421c4d72b1223f560f136e46
I20260811 11:56:02.022722 15726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:56:02.031575 15726 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:56:02.032526 15726 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:56:02.040349 15726 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee: Generated new TSK 0
I20260811 11:56:02.040979 15726 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
I20260811 11:56:02.070866 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:0
--local_ip_for_outbound_sockets=127.6.143.129
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:46709
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:34565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:56:02.184181 15741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:56:02.184422 15741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:56:02.184464 15741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:56:02.188438 15741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:56:02.188490 15741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:56:02.188546 15741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:56:02.193480 15741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:34565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:46709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:56:02.194427 15741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:56:02.195360 15741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:56:02.204617 15746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:56:02.204779 15741 server_base.cc:1034] running on GCE node
W20260811 11:56:02.204622 15749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:02.204622 15747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:56:02.205418 15741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:56:02.206038 15741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:56:02.207224 15741 hybrid_clock.cc:648] HybridClock initialized: now 1786449362207208 us; error 25 us; skew 500 ppm
Aug 11 11:56:02 dist-test-slave-zkpd krb5kdc[15615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449362, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:56:02.210017 15741 init.cc:377] Logged in from keytab as kudu/127.6.143.129@KRBTEST.COM (short username kudu)
I20260811 11:56:02.210925 15741 webserver.cc:460] Webserver started at http://127.6.143.129:34645/ using document root <none> and password file <none>
I20260811 11:56:02.211526 15741 fs_manager.cc:359] Metadata directory not provided
I20260811 11:56:02.211619 15741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:56:02.211835 15741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:56:02.213965 15741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/data/instance:
uuid: "79c4ac5c09a8480ab88cbb3bd4d17e29"
format_stamp: "Formatted at 2026-08-11 11:56:02 on dist-test-slave-zkpd"
server_key: "b2bdffd655d9bcc0608c9dc8c1d915bb"
server_key_iv: "6092083b4fde86f4f351f7a9464c8615"
server_key_version: "encryptionkey@0"
I20260811 11:56:02.214558 15741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance:
uuid: "79c4ac5c09a8480ab88cbb3bd4d17e29"
format_stamp: "Formatted at 2026-08-11 11:56:02 on dist-test-slave-zkpd"
server_key: "b2bdffd655d9bcc0608c9dc8c1d915bb"
server_key_iv: "6092083b4fde86f4f351f7a9464c8615"
server_key_version: "encryptionkey@0"
I20260811 11:56:02.218569 15741 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 11:56:02.221109 15756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:02.222379 15741 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 11:56:02.222532 15741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "79c4ac5c09a8480ab88cbb3bd4d17e29"
format_stamp: "Formatted at 2026-08-11 11:56:02 on dist-test-slave-zkpd"
server_key: "b2bdffd655d9bcc0608c9dc8c1d915bb"
server_key_iv: "6092083b4fde86f4f351f7a9464c8615"
server_key_version: "encryptionkey@0"
I20260811 11:56:02.222648 15741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:56:02.247931 15741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:56:02.250682 15741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:56:02.250905 15741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:56:02.251482 15741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:56:02.252374 15741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:56:02.252444 15741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:02.252516 15741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:56:02.252559 15741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:02.261374 15741 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:34757
I20260811 11:56:02.261410 15882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:34757 every 8 connection(s)
I20260811 11:56:02.262329 15741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:56:02.267757 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 15741
I20260811 11:56:02.267887 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance
I20260811 11:56:02.268208 6718 external_mini_cluster.cc:1428] Setting key 9897d5fc7ff396ea4aa6b7e2ebf33f91
Aug 11 11:56:02 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449362, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:56:02.276774 15715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:02.264133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:59497 (local address 127.6.143.190:46709)
0811 11:56:02.264280 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:02.264287 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:02.265025 (+ 738us) server_negotiation.cc:407] Connection header received
0811 11:56:02.266297 (+ 1272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:02.266300 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:02.266363 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:02.266458 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:02.267619 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:02.268350 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:02.269538 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:02.269836 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:02.269948 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:02.272655 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:56:02.272658 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:56:02.272660 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:56:02.272684 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:56:02.274452 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:56:02.274960 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:56:02.274967 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:56:02.274971 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:56:02.275076 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:56:02.275457 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:56:02.275461 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:56:02.275463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:56:02.275712 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:56:02.275827 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 11:56:02.276369 (+ 542us) server_negotiation.cc:299] Negotiation successful
0811 11:56:02.276528 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 11:56:02.277351 15885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:02.264441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.129:59497)
0811 11:56:02.264875 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:02.264923 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:02.266083 (+ 1160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:02.266634 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:02.266641 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:02.266988 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:02.267442 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:02.267456 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:02.268481 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:02.268485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:02.269395 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:02.269407 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:02.269992 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:02.269995 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:02.270143 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:02.270726 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:02.270742 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:02.272480 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:56:02.274572 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:56:02.274577 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:56:02.274588 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:56:02.274838 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:56:02.275196 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:56:02.275205 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:56:02.275207 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:56:02.275340 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:56:02.275821 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:56:02.275826 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:56:02.276118 (+ 292us) client_negotiation.cc:769] Sending connection context
0811 11:56:02.276276 (+ 158us) client_negotiation.cc:240] Negotiation successful
0811 11:56:02.276529 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":101,"threads_started":1}
I20260811 11:56:02.278613 15883 heartbeater.cc:347] Connected to a master server at 127.6.143.190:46709
I20260811 11:56:02.278895 15883 heartbeater.cc:464] Registering TS with master...
I20260811 11:56:02.279511 15883 heartbeater.cc:511] Master 127.6.143.190:46709 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.143.130@KRBTEST.COM; defaulting to no policy
I20260811 11:56:02.281234 15669 ts_manager.cc:194] Registered new tserver with Master: 79c4ac5c09a8480ab88cbb3bd4d17e29 (127.6.143.129:34757)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.130@KRBTEST.COM" created.
I20260811 11:56:02.282648 15669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.129@KRBTEST.COM'} at 127.6.143.129:59497
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
I20260811 11:56:02.298846 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:0
--local_ip_for_outbound_sockets=127.6.143.130
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:46709
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:34565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:56:02.414561 15888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:56:02.414829 15888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:56:02.414912 15888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:56:02.419049 15888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:56:02.419142 15888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:56:02.419226 15888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:56:02.424394 15888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:34565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:46709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:56:02.425386 15888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:56:02.426374 15888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:56:02.435730 15894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:02.435727 15895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:02.435719 15897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:56:02.436040 15888 server_base.cc:1034] running on GCE node
I20260811 11:56:02.436425 15888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:56:02.437012 15888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:56:02.438199 15888 hybrid_clock.cc:648] HybridClock initialized: now 1786449362438183 us; error 26 us; skew 500 ppm
Aug 11 11:56:02 dist-test-slave-zkpd krb5kdc[15615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449362, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:56:02.445956 15888 init.cc:377] Logged in from keytab as kudu/127.6.143.130@KRBTEST.COM (short username kudu)
I20260811 11:56:02.446822 15888 webserver.cc:460] Webserver started at http://127.6.143.130:45451/ using document root <none> and password file <none>
I20260811 11:56:02.447369 15888 fs_manager.cc:359] Metadata directory not provided
I20260811 11:56:02.447429 15888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:56:02.447664 15888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:56:02.449712 15888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/data/instance:
uuid: "7d79024b64be45af81020c822b9a6cbe"
format_stamp: "Formatted at 2026-08-11 11:56:02 on dist-test-slave-zkpd"
server_key: "e4ed4a5bd6d29df6dd22887823a9c7e9"
server_key_iv: "1697e59367d28540901ae072fed79e7d"
server_key_version: "encryptionkey@0"
I20260811 11:56:02.450269 15888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance:
uuid: "7d79024b64be45af81020c822b9a6cbe"
format_stamp: "Formatted at 2026-08-11 11:56:02 on dist-test-slave-zkpd"
server_key: "e4ed4a5bd6d29df6dd22887823a9c7e9"
server_key_iv: "1697e59367d28540901ae072fed79e7d"
server_key_version: "encryptionkey@0"
I20260811 11:56:02.453899 15888 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 11:56:02.456257 15904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:02.457473 15888 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 11:56:02.457614 15888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "7d79024b64be45af81020c822b9a6cbe"
format_stamp: "Formatted at 2026-08-11 11:56:02 on dist-test-slave-zkpd"
server_key: "e4ed4a5bd6d29df6dd22887823a9c7e9"
server_key_iv: "1697e59367d28540901ae072fed79e7d"
server_key_version: "encryptionkey@0"
I20260811 11:56:02.457742 15888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:56:02.478698 15888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:56:02.482226 15888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:56:02.482429 15888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:56:02.483033 15888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:56:02.483877 15888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:56:02.483970 15888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:02.484043 15888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:56:02.484097 15888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:02.492628 15888 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:46729
I20260811 11:56:02.492663 16041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:46729 every 8 connection(s)
I20260811 11:56:02.493474 15888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
I20260811 11:56:02.495759 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 15888
I20260811 11:56:02.495942 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance
I20260811 11:56:02.496316 6718 external_mini_cluster.cc:1428] Setting key cec76071fcf8b7dcf708a2520983edc3
Aug 11 11:56:02 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449362, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.143.131@KRBTEST.COM; defaulting to no policy
I20260811 11:56:02.507944 15715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:02.495266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:47723 (local address 127.6.143.190:46709)
0811 11:56:02.495416 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:02.495421 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:02.496415 (+ 994us) server_negotiation.cc:407] Connection header received
0811 11:56:02.498034 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:02.498037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:02.498078 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:02.498136 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:02.499764 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:02.500345 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:02.501479 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:02.501700 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:02.501813 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:02.504480 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:56:02.504483 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:56:02.504485 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:56:02.504506 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:56:02.505925 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:56:02.506431 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:56:02.506435 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:56:02.506437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:56:02.506531 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:56:02.506877 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:56:02.506880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:56:02.506882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:56:02.507095 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:56:02.507197 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 11:56:02.507646 (+ 449us) server_negotiation.cc:299] Negotiation successful
0811 11:56:02.507761 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 11:56:02.508785 16044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:02.495669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.130:47723)
0811 11:56:02.496241 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:02.496299 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:02.497774 (+ 1475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:02.498325 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:02.498337 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:02.498875 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:02.499596 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:02.499620 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:02.500473 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:02.500476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:02.501351 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:02.501361 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:02.501857 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:02.501861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:02.502029 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:02.502625 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:02.502654 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:02.504305 (+ 1651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:56:02.506042 (+ 1737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:56:02.506049 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:56:02.506062 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:56:02.506324 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:56:02.506629 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:56:02.506632 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:56:02.506634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:56:02.506774 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:56:02.507196 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:56:02.507202 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:56:02.507425 (+ 223us) client_negotiation.cc:769] Sending connection context
0811 11:56:02.507604 (+ 179us) client_negotiation.cc:240] Negotiation successful
0811 11:56:02.507944 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":171,"threads_started":1}
I20260811 11:56:02.510072 16042 heartbeater.cc:347] Connected to a master server at 127.6.143.190:46709
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.131@KRBTEST.COM" created.
I20260811 11:56:02.510341 16042 heartbeater.cc:464] Registering TS with master...
I20260811 11:56:02.510844 16042 heartbeater.cc:511] Master 127.6.143.190:46709 requested a full tablet report, sending...
I20260811 11:56:02.511996 15669 ts_manager.cc:194] Registered new tserver with Master: 7d79024b64be45af81020c822b9a6cbe (127.6.143.130:46729)
I20260811 11:56:02.512557 15669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.130@KRBTEST.COM'} at 127.6.143.130:47723
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
I20260811 11:56:02.525060 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:0
--local_ip_for_outbound_sockets=127.6.143.131
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:46709
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:34565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:56:02.637816 16047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:56:02.638094 16047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:56:02.638170 16047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:56:02.642136 16047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:56:02.642220 16047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:56:02.642310 16047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:56:02.647385 16047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:34565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:46709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:56:02.648443 16047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:56:02.649439 16047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:56:02.658989 16053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:02.658989 16056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:02.659017 16054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:56:02.659211 16047 server_base.cc:1034] running on GCE node
I20260811 11:56:02.659778 16047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:56:02.660486 16047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:56:02.661684 16047 hybrid_clock.cc:648] HybridClock initialized: now 1786449362661657 us; error 53 us; skew 500 ppm
Aug 11 11:56:02 dist-test-slave-zkpd krb5kdc[15615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449362, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:56:02.664674 16047 init.cc:377] Logged in from keytab as kudu/127.6.143.131@KRBTEST.COM (short username kudu)
I20260811 11:56:02.665576 16047 webserver.cc:460] Webserver started at http://127.6.143.131:37763/ using document root <none> and password file <none>
I20260811 11:56:02.666154 16047 fs_manager.cc:359] Metadata directory not provided
I20260811 11:56:02.666214 16047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:56:02.666461 16047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:56:02.668601 16047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/data/instance:
uuid: "56db8932b3e14ace9f0dfe782cfe5174"
format_stamp: "Formatted at 2026-08-11 11:56:02 on dist-test-slave-zkpd"
server_key: "3573da9567cd58109acd514b88668778"
server_key_iv: "05b60c21d93b29c9c0ee7947dd01503b"
server_key_version: "encryptionkey@0"
I20260811 11:56:02.669174 16047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance:
uuid: "56db8932b3e14ace9f0dfe782cfe5174"
format_stamp: "Formatted at 2026-08-11 11:56:02 on dist-test-slave-zkpd"
server_key: "3573da9567cd58109acd514b88668778"
server_key_iv: "05b60c21d93b29c9c0ee7947dd01503b"
server_key_version: "encryptionkey@0"
I20260811 11:56:02.672828 16047 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 11:56:02.675174 16066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:02.676425 16047 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:56:02.676558 16047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "56db8932b3e14ace9f0dfe782cfe5174"
format_stamp: "Formatted at 2026-08-11 11:56:02 on dist-test-slave-zkpd"
server_key: "3573da9567cd58109acd514b88668778"
server_key_iv: "05b60c21d93b29c9c0ee7947dd01503b"
server_key_version: "encryptionkey@0"
I20260811 11:56:02.676664 16047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:56:02.690300 16047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:56:02.693486 16047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:56:02.693693 16047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:56:02.694264 16047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:56:02.695079 16047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:56:02.695151 16047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:02.695222 16047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:56:02.695276 16047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:02.704643 16047 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:39699
I20260811 11:56:02.704687 16198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:39699 every 8 connection(s)
I20260811 11:56:02.705485 16047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:56:02.711618 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 16047
I20260811 11:56:02.711772 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance
I20260811 11:56:02.712124 6718 external_mini_cluster.cc:1428] Setting key 1f59f0bf4de7723ab0e77b61a24cad52
Aug 11 11:56:02 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449362, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:56:02.719370 15715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:02.707267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:48613 (local address 127.6.143.190:46709)
0811 11:56:02.707421 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:02.707425 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:02.708240 (+ 815us) server_negotiation.cc:407] Connection header received
0811 11:56:02.709482 (+ 1242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:02.709485 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:02.709534 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:02.709606 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:02.710729 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:02.711254 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:02.712574 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:02.712844 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:02.712956 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:02.715651 (+ 2695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:56:02.715655 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:56:02.715658 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:56:02.715686 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:56:02.717282 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:56:02.717803 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:56:02.717807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:56:02.717809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:56:02.717906 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:56:02.718257 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:56:02.718262 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:56:02.718263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:56:02.718463 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:56:02.718573 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 11:56:02.719071 (+ 498us) server_negotiation.cc:299] Negotiation successful
0811 11:56:02.719199 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 11:56:02.720232 16201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:02.707562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.131:48613)
0811 11:56:02.708086 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:02.708128 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:02.709268 (+ 1140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:02.709763 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:02.709769 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:02.710117 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:02.710584 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:02.710615 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:02.711361 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:02.711364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:02.712434 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:02.712445 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:02.712993 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:02.712997 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:02.713231 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:02.713919 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:02.713936 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:02.715491 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:56:02.717412 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:56:02.717418 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:56:02.717433 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:56:02.717693 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:56:02.718002 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:56:02.718005 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:56:02.718007 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:56:02.718158 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:56:02.718572 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:56:02.718581 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:56:02.718828 (+ 247us) client_negotiation.cc:769] Sending connection context
0811 11:56:02.719027 (+ 199us) client_negotiation.cc:240] Negotiation successful
0811 11:56:02.719348 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":109,"threads_started":1}
I20260811 11:56:02.721393 16199 heartbeater.cc:347] Connected to a master server at 127.6.143.190:46709
I20260811 11:56:02.721661 16199 heartbeater.cc:464] Registering TS with master...
I20260811 11:56:02.722160 16199 heartbeater.cc:511] Master 127.6.143.190:46709 requested a full tablet report, sending...
I20260811 11:56:02.723186 15669 ts_manager.cc:194] Registered new tserver with Master: 56db8932b3e14ace9f0dfe782cfe5174 (127.6.143.131:39699)
I20260811 11:56:02.723759 15669 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.131@KRBTEST.COM'} at 127.6.143.131:48613
I20260811 11:56:02.726014 6718 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.6.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.129.keytab.
Entry for principal oryx/127.6.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.129.keytab.
WARNING: no policy specified for oryx/127.6.143.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.143.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.130.keytab.
Entry for principal oryx/127.6.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.130.keytab.
WARNING: no policy specified for oryx/127.6.143.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.131.keytab.
Entry for principal oryx/127.6.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.131.keytab.
I20260811 11:56:02.797880 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 15741
I20260811 11:56:02.807186 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 15888
I20260811 11:56:02.815971 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 16047
I20260811 11:56:02.826738 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 15631
I20260811 11:56:02.835691 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:46709
--webserver_interface=127.6.143.190
--webserver_port=45549
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:34565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:46709
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:56:02.947539 16209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:56:02.947840 16209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:56:02.947954 16209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:56:02.951858 16209 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:56:02.951967 16209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:56:02.952008 16209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:56:02.952044 16209 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:56:02.952075 16209 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:56:02.957219 16209 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:34565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:46709
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:46709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=45549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:56:02.958238 16209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:56:02.959257 16209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:56:02.967761 16219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:02.967801 16221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:02.967849 16218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:56:02.968109 16209 server_base.cc:1034] running on GCE node
I20260811 11:56:02.968676 16209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:56:02.969766 16209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:56:02.970953 16209 hybrid_clock.cc:648] HybridClock initialized: now 1786449362970919 us; error 47 us; skew 500 ppm
Aug 11 11:56:02 dist-test-slave-zkpd krb5kdc[15615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449362, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:56:02.973881 16209 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:56:02.974733 16209 webserver.cc:460] Webserver started at http://127.6.143.190:45549/ using document root <none> and password file <none>
I20260811 11:56:02.975310 16209 fs_manager.cc:359] Metadata directory not provided
I20260811 11:56:02.975428 16209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:56:02.979224 16209 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:56:02.981343 16228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:02.982518 16209 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 11:56:02.982679 16209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "824caff9b6a44cc0ba1acd316ec815ee"
format_stamp: "Formatted at 2026-08-11 11:56:01 on dist-test-slave-zkpd"
server_key: "4241a3fa61c742e56b8b079a9141fb21"
server_key_iv: "a6f7dd12e1fad1f5047fadc1a7ef584f"
server_key_version: "encryptionkey@0"
I20260811 11:56:02.983210 16209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:56:03.001204 16209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:56:03.004472 16209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:56:03.004675 16209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:56:03.011813 16209 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:46709
I20260811 11:56:03.011842 16289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:46709 every 8 connection(s)
I20260811 11:56:03.012876 16209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:56:03.017179 16290 sys_catalog.cc:263] Verifying existing consensus state
I20260811 11:56:03.018821 16290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee: Bootstrap starting.
I20260811 11:56:03.022656 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 16209
I20260811 11:56:03.023099 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:34757
--local_ip_for_outbound_sockets=127.6.143.129
--tserver_master_addrs=127.6.143.190:46709
--webserver_port=34645
--webserver_interface=127.6.143.129
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:34565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.143.129
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:56:03.039557 16290 log.cc:824] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee: Log is configured to *not* fsync() on all Append() calls
I20260811 11:56:03.048066 16290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 11:56:03.048473 16290 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee: Bootstrap complete.
I20260811 11:56:03.054315 16290 raft_consensus.cc:348] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824caff9b6a44cc0ba1acd316ec815ee" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46709 } }
I20260811 11:56:03.054826 16290 raft_consensus.cc:729] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 824caff9b6a44cc0ba1acd316ec815ee, State: Initialized, Role: FOLLOWER
I20260811 11:56:03.055373 16290 consensus_queue.cc:260] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824caff9b6a44cc0ba1acd316ec815ee" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46709 } }
I20260811 11:56:03.055524 16290 raft_consensus.cc:388] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:56:03.055593 16290 raft_consensus.cc:482] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:56:03.055685 16290 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [term 1 FOLLOWER]: Advancing to term 2
I20260811 11:56:03.058151 16290 raft_consensus.cc:504] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824caff9b6a44cc0ba1acd316ec815ee" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46709 } }
I20260811 11:56:03.058532 16290 leader_election.cc:302] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 824caff9b6a44cc0ba1acd316ec815ee; no voters:
I20260811 11:56:03.058861 16290 leader_election.cc:288] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 11:56:03.058995 16294 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [term 2 FOLLOWER]: Leader election won for term 2
I20260811 11:56:03.059284 16294 raft_consensus.cc:686] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [term 2 LEADER]: Becoming Leader. State: Replica: 824caff9b6a44cc0ba1acd316ec815ee, State: Running, Role: LEADER
I20260811 11:56:03.059653 16294 consensus_queue.cc:237] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824caff9b6a44cc0ba1acd316ec815ee" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46709 } }
I20260811 11:56:03.060148 16290 sys_catalog.cc:564] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:56:03.061244 16295 sys_catalog.cc:455] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "824caff9b6a44cc0ba1acd316ec815ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824caff9b6a44cc0ba1acd316ec815ee" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46709 } } }
I20260811 11:56:03.061417 16295 sys_catalog.cc:458] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [sys.catalog]: This master's current role is: LEADER
I20260811 11:56:03.061244 16296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [sys.catalog]: SysCatalogTable state changed. Reason: New leader 824caff9b6a44cc0ba1acd316ec815ee. Latest consensus state: current_term: 2 leader_uuid: "824caff9b6a44cc0ba1acd316ec815ee" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "824caff9b6a44cc0ba1acd316ec815ee" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46709 } } }
I20260811 11:56:03.061486 16296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee [sys.catalog]: This master's current role is: LEADER
I20260811 11:56:03.062114 16305 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:56:03.065332 16305 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:56:03.065979 16305 catalog_manager.cc:1259] Loaded cluster ID: ba84c562421c4d72b1223f560f136e46
I20260811 11:56:03.066042 16305 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:56:03.068436 16305 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:56:03.069130 16305 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 824caff9b6a44cc0ba1acd316ec815ee: Loaded TSK: 0
I20260811 11:56:03.070331 16305 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 11:56:03.147786 16292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:56:03.148092 16292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:56:03.148186 16292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:56:03.152288 16292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:56:03.152365 16292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:56:03.152457 16292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:56:03.157567 16292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:34565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.129.keytab
--principal=oryx/127.6.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:34757
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=34645
--tserver_master_addrs=127.6.143.190:46709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:56:03.158600 16292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:56:03.159631 16292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:56:03.170682 16323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:03.170802 16319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:03.170919 16318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:56:03.171736 16292 server_base.cc:1034] running on GCE node
I20260811 11:56:03.172173 16292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:56:03.172806 16292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:56:03.173998 16292 hybrid_clock.cc:648] HybridClock initialized: now 1786449363173968 us; error 49 us; skew 500 ppm
Aug 11 11:56:03 dist-test-slave-zkpd krb5kdc[15615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449363, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:56:03.177208 16292 init.cc:377] Logged in from keytab as oryx/127.6.143.129@KRBTEST.COM (short username oryx)
I20260811 11:56:03.178159 16292 webserver.cc:460] Webserver started at http://127.6.143.129:34645/ using document root <none> and password file <none>
I20260811 11:56:03.178722 16292 fs_manager.cc:359] Metadata directory not provided
I20260811 11:56:03.178794 16292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:56:03.182873 16292 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 11:56:03.185036 16331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:03.186363 16292 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:56:03.186482 16292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "79c4ac5c09a8480ab88cbb3bd4d17e29"
format_stamp: "Formatted at 2026-08-11 11:56:02 on dist-test-slave-zkpd"
server_key: "b2bdffd655d9bcc0608c9dc8c1d915bb"
server_key_iv: "6092083b4fde86f4f351f7a9464c8615"
server_key_version: "encryptionkey@0"
I20260811 11:56:03.187139 16292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:56:03.218305 16292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:56:03.221756 16292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:56:03.221920 16292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:56:03.222564 16292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:56:03.223393 16292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:56:03.223444 16292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:03.223515 16292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:56:03.223547 16292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:03.232797 16292 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:34757
I20260811 11:56:03.232837 16458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:34757 every 8 connection(s)
I20260811 11:56:03.233774 16292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:56:03.241559 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 16292
I20260811 11:56:03.242046 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:46729
--local_ip_for_outbound_sockets=127.6.143.130
--tserver_master_addrs=127.6.143.190:46709
--webserver_port=45451
--webserver_interface=127.6.143.130
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:34565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.143.130
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 11:56:03 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:03 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:03.246713 16462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:03.235941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.129:59407)
0811 11:56:03.236419 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:03.236470 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:03.237639 (+ 1169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:03.238650 (+ 1011us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:03.238657 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:03.238996 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:03.239426 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:03.239438 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.240747 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.240749 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:03.241810 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:03.241824 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.242689 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.242697 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:03.242930 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:03.243520 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:03.243546 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:03.246030 (+ 2484us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":131,"threads_started":1}
W20260811 11:56:03.247274 16461 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:03.235827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:59407 (local address 127.6.143.190:46709)
0811 11:56:03.236318 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:03.236337 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:03.236582 (+ 245us) server_negotiation.cc:407] Connection header received
0811 11:56:03.237995 (+ 1413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:03.238006 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:03.238287 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:03.238559 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:03.239576 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.240623 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.241949 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.242520 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.242643 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:03.246331 (+ 3688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.129:59407: BlockingRecv error: recv got EOF from 127.6.143.129:59407 (error 108)
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":130,"threads_started":1}
W20260811 11:56:03.247315 16459 heartbeater.cc:644] Failed to heartbeat to 127.6.143.190:46709 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.143.190:46709: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Aug 11 11:56:03 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:03 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:03.253626 16462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:03.247747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.129:42735)
0811 11:56:03.247999 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:03.248018 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:03.248105 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:03.248354 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:03.248359 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:03.248544 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:03.248683 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:03.248691 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.249390 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.249396 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:03.250433 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:03.250449 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.250864 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.250868 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:03.250996 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:03.251456 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:03.251476 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:03.253459 (+ 1983us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":159}
W20260811 11:56:03.253649 16461 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:03.247950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:42735 (local address 127.6.143.190:46709)
0811 11:56:03.248049 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:03.248052 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:03.248064 (+ 12us) server_negotiation.cc:407] Connection header received
0811 11:56:03.248198 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:03.248201 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:03.248236 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:03.248303 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:03.248805 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.249282 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.250554 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.250752 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.250825 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:03.253570 (+ 2745us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.129:42735: BlockingRecv error: recv got EOF from 127.6.143.129:42735 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 11 11:56:03 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:03 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:03.259025 16462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:03.254185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.129:34197)
0811 11:56:03.254313 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:03.254328 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:03.254421 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:03.254668 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:03.254671 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:03.254818 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:03.254881 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:03.254885 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.255568 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.255572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:03.256226 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:03.256237 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.256822 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.256825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:03.256882 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:03.257319 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:03.257349 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:03.258946 (+ 1597us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 11:56:03.259183 16461 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:03.254246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:34197 (local address 127.6.143.190:46709)
0811 11:56:03.254348 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:03.254354 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:03.254369 (+ 15us) server_negotiation.cc:407] Connection header received
0811 11:56:03.254520 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:03.254524 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:03.254571 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:03.254654 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:03.254991 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.255461 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.256369 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.256713 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.256812 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:03.259085 (+ 2273us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.129:34197: BlockingRecv error: recv got EOF from 127.6.143.129:34197 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260811 11:56:03.259213 16459 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 11:56:03.359439 16464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:56:03.359681 16464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:56:03.359728 16464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:56:03.363776 16464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:56:03.363831 16464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:56:03.363932 16464 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:56:03.369076 16464 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:34565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.130.keytab
--principal=oryx/127.6.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:46729
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=45451
--tserver_master_addrs=127.6.143.190:46709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16464
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:56:03.369961 16464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:56:03.370885 16464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:56:03.381534 16470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:03.381527 16471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:03.381527 16473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:56:03.381713 16464 server_base.cc:1034] running on GCE node
I20260811 11:56:03.382288 16464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:56:03.382942 16464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:56:03.384130 16464 hybrid_clock.cc:648] HybridClock initialized: now 1786449363384114 us; error 25 us; skew 500 ppm
Aug 11 11:56:03 dist-test-slave-zkpd krb5kdc[15615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449363, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:56:03.386976 16464 init.cc:377] Logged in from keytab as oryx/127.6.143.130@KRBTEST.COM (short username oryx)
I20260811 11:56:03.388017 16464 webserver.cc:460] Webserver started at http://127.6.143.130:45451/ using document root <none> and password file <none>
I20260811 11:56:03.388581 16464 fs_manager.cc:359] Metadata directory not provided
I20260811 11:56:03.388686 16464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:56:03.392463 16464 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260811 11:56:03.394500 16480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:03.395686 16464 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:56:03.395830 16464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "7d79024b64be45af81020c822b9a6cbe"
format_stamp: "Formatted at 2026-08-11 11:56:02 on dist-test-slave-zkpd"
server_key: "e4ed4a5bd6d29df6dd22887823a9c7e9"
server_key_iv: "1697e59367d28540901ae072fed79e7d"
server_key_version: "encryptionkey@0"
I20260811 11:56:03.396420 16464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:56:03.426558 16464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:56:03.429503 16464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:56:03.429723 16464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:56:03.430326 16464 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:56:03.431143 16464 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:56:03.431216 16464 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:03.431304 16464 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:56:03.431365 16464 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:03.440361 16464 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:46729
I20260811 11:56:03.440410 16606 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:46729 every 8 connection(s)
I20260811 11:56:03.441351 16464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
I20260811 11:56:03.450331 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 16464
Aug 11 11:56:03 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:03.450842 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:39699
--local_ip_for_outbound_sockets=127.6.143.131
--tserver_master_addrs=127.6.143.190:46709
--webserver_port=37763
--webserver_interface=127.6.143.131
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:34565
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.143.131
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 11:56:03 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 11:56:03.452600 16461 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:03.443135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:54229 (local address 127.6.143.190:46709)
0811 11:56:03.443283 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:03.443287 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:03.444105 (+ 818us) server_negotiation.cc:407] Connection header received
0811 11:56:03.445407 (+ 1302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:03.445411 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:03.445473 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:03.445569 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:03.446731 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.447262 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.448507 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.448739 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.448853 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:03.452501 (+ 3648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.130:54229: BlockingRecv error: recv got EOF from 127.6.143.130:54229 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 11:56:03.453290 16610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:03.443501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.130:54229)
0811 11:56:03.443957 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:03.443998 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:03.445180 (+ 1182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:03.445749 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:03.445756 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:03.446094 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:03.446554 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:03.446567 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.447393 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.447399 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:03.448379 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:03.448389 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.448888 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.448892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:03.449057 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:03.449641 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:03.449658 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:03.452386 (+ 2728us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":108,"threads_started":1}
W20260811 11:56:03.453742 16608 heartbeater.cc:644] Failed to heartbeat to 127.6.143.190:46709 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.143.190:46709: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Aug 11 11:56:03 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:03 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:03.460754 16610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:03.454198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.130:40551)
0811 11:56:03.454298 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:03.454314 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:03.454392 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:03.454677 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:03.454682 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:03.454862 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:03.455012 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:03.455023 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.455701 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.455707 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:03.456765 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:03.456780 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.457228 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.457235 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:03.457373 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:03.457817 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:03.457836 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:03.460418 (+ 2582us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 11:56:03.460779 16461 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:03.454278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:40551 (local address 127.6.143.190:46709)
0811 11:56:03.454386 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:03.454390 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:03.454415 (+ 25us) server_negotiation.cc:407] Connection header received
0811 11:56:03.454516 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:03.454518 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:03.454562 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:03.454631 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:03.455131 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.455590 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.456903 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.457109 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.457189 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:03.460574 (+ 3385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.130:40551: BlockingRecv error: recv got EOF from 127.6.143.130:40551 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 11 11:56:03 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:03 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:03.468321 16610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:03.462104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.130:33097)
0811 11:56:03.462447 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:03.462465 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:03.462565 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:03.462947 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:03.462950 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:03.463075 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:03.463142 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:03.463147 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.464263 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.464266 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:03.465246 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:03.465263 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.465901 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.465904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:03.465982 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:03.466485 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:03.466499 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:03.468189 (+ 1690us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255}
W20260811 11:56:03.468420 16461 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:03.462274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:33097 (local address 127.6.143.190:46709)
0811 11:56:03.462434 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:03.462439 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:03.462535 (+ 96us) server_negotiation.cc:407] Connection header received
0811 11:56:03.462665 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:03.462667 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:03.462718 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:03.462802 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:03.463258 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.464058 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.465410 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.465787 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.465899 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:03.468298 (+ 2399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.130:33097: BlockingRecv error: recv got EOF from 127.6.143.130:33097 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 11:56:03.468519 16608 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 11:56:03.568462 16611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:56:03.568718 16611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:56:03.568802 16611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:56:03.572767 16611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:56:03.572839 16611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:56:03.572933 16611 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:56:03.578107 16611 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:34565
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/krb5kdc/oryx_127.6.143.131.keytab
--principal=oryx/127.6.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:39699
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=37763
--tserver_master_addrs=127.6.143.190:46709
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:56:03.579142 16611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:56:03.580199 16611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 11:56:03.589726 16611 server_base.cc:1034] running on GCE node
W20260811 11:56:03.589751 16616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:03.589753 16617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:03.589882 16619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:56:03.590386 16611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:56:03.591087 16611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:56:03.592267 16611 hybrid_clock.cc:648] HybridClock initialized: now 1786449363592230 us; error 49 us; skew 500 ppm
Aug 11 11:56:03 dist-test-slave-zkpd krb5kdc[15615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449363, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:56:03.594928 16611 init.cc:377] Logged in from keytab as oryx/127.6.143.131@KRBTEST.COM (short username oryx)
I20260811 11:56:03.595781 16611 webserver.cc:460] Webserver started at http://127.6.143.131:37763/ using document root <none> and password file <none>
I20260811 11:56:03.596364 16611 fs_manager.cc:359] Metadata directory not provided
I20260811 11:56:03.596484 16611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:56:03.600238 16611 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 11:56:03.602140 16627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:03.603319 16611 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:56:03.603471 16611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "56db8932b3e14ace9f0dfe782cfe5174"
format_stamp: "Formatted at 2026-08-11 11:56:02 on dist-test-slave-zkpd"
server_key: "3573da9567cd58109acd514b88668778"
server_key_iv: "05b60c21d93b29c9c0ee7947dd01503b"
server_key_version: "encryptionkey@0"
I20260811 11:56:03.604032 16611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:56:03.618203 16611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:56:03.621352 16611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:56:03.621549 16611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:56:03.622126 16611 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:56:03.622970 16611 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:56:03.623041 16611 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:03.623111 16611 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:56:03.623152 16611 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:03.632680 16611 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:39699
I20260811 11:56:03.632732 16752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:39699 every 8 connection(s)
I20260811 11:56:03.633663 16611 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:56:03.639282 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 16611
Aug 11 11:56:03 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:03 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 11:56:03.646023 16461 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:03.635480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:54481 (local address 127.6.143.190:46709)
0811 11:56:03.635623 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:03.635627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:03.636487 (+ 860us) server_negotiation.cc:407] Connection header received
0811 11:56:03.637835 (+ 1348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:03.637838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:03.637899 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:03.638007 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:03.639267 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.640095 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.641493 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.641789 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.641870 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:03.645921 (+ 4051us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.131:54481: BlockingRecv error: recv got EOF from 127.6.143.131:54481 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 11:56:03.646796 16755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:03.635852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.131:54481)
0811 11:56:03.636341 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:03.636384 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:03.637612 (+ 1228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:03.638186 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:03.638193 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:03.638542 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:03.639077 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:03.639100 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.640235 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.640238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:03.641367 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:03.641378 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.641943 (+ 565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.641947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:03.642136 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:03.642788 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:03.642805 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:03.645762 (+ 2957us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":130,"threads_started":1}
W20260811 11:56:03.647269 16753 heartbeater.cc:644] Failed to heartbeat to 127.6.143.190:46709 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.143.190:46709: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
I20260811 11:56:03.649778 16758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:03.640418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52098 (local address 127.6.143.190:46709)
0811 11:56:03.640688 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:03.640692 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:03.640705 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:56:03.640742 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:03.640744 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:03.640785 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:03.640894 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:03.641591 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.642256 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.643186 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.643569 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.643652 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:03.645232 (+ 1580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:56:03.645242 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:56:03.645264 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:56:03.645300 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:56:03.647388 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:56:03.647972 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:56:03.647980 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:56:03.647990 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:56:03.648158 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:56:03.648484 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:56:03.648488 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:56:03.648490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:56:03.648897 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:56:03.649042 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 11:56:03.649362 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 11:56:03.649615 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":111,"threads_started":1}
Aug 11 11:56:03 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:03 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:03.653069 16755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:03.647671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.131:39327)
0811 11:56:03.647951 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:03.647966 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:03.648051 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:03.648275 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:03.648278 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:03.648417 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:03.648554 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:03.648562 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.649362 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.649365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:03.650100 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:03.650110 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.650557 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.650560 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:03.650653 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:03.651111 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:03.651132 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:03.652934 (+ 1802us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173}
W20260811 11:56:03.653159 16461 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:03.647785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:39327 (local address 127.6.143.190:46709)
0811 11:56:03.647943 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:03.647947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:03.648008 (+ 61us) server_negotiation.cc:407] Connection header received
0811 11:56:03.648134 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:03.648136 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:03.648169 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:03.648256 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:03.648667 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.649256 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.650221 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.650445 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.650554 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:03.653041 (+ 2487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.131:39327: BlockingRecv error: recv got EOF from 127.6.143.131:39327 (error 108)
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":49}
Aug 11 11:56:03 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:03 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:03.658064 16755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:03.653591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.131:52907)
0811 11:56:03.653723 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:03.653739 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:03.653815 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:03.653994 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:03.653996 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:03.654113 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:03.654178 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:03.654183 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.654818 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.654820 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:03.655321 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:03.655328 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.655760 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.655763 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:03.655827 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:03.656358 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:03.656370 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:03.657975 (+ 1605us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 11:56:03.658241 16461 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:03.653667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:52907 (local address 127.6.143.190:46709)
0811 11:56:03.653821 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:03.653824 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:03.653836 (+ 12us) server_negotiation.cc:407] Connection header received
0811 11:56:03.653869 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:03.653871 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:03.653902 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:03.653964 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:03.654271 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.654732 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.655419 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:03.655616 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:03.655690 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:03.658089 (+ 2399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.131:52907: BlockingRecv error: recv got EOF from 127.6.143.131:52907 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260811 11:56:03.658317 16753 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 11:56:04 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:04 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:04.265745 16759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:04.259988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.129:57985)
0811 11:56:04.260263 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:04.260276 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:04.260380 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:04.260643 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:04.260646 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:04.260842 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:04.260943 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:04.260948 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:04.261972 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:04.261992 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:04.262755 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:04.262766 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:04.263181 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:04.263183 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:04.263243 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:04.263716 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:04.263727 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:04.265627 (+ 1900us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":84,"threads_started":1}
W20260811 11:56:04.265944 16760 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:04.260047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:57985 (local address 127.6.143.190:46709)
0811 11:56:04.260319 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:04.260323 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:04.260336 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:56:04.260472 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:04.260475 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:04.260519 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:04.260599 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:04.261085 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:04.261853 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:04.262863 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:04.263085 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:04.263167 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:04.265758 (+ 2591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.129:57985: BlockingRecv error: recv got EOF from 127.6.143.129:57985 (error 108)
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":93,"threads_started":1}
Aug 11 11:56:04 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:04 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:04.474531 16761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:04.469167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.130:44959)
0811 11:56:04.469496 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:04.469509 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:04.469609 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:04.469947 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:04.469951 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:04.470152 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:04.470274 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:04.470280 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:04.471013 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:04.471015 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:04.471593 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:04.471601 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:04.472061 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:04.472064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:04.472139 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:04.472615 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:04.472626 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:04.474392 (+ 1766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":86,"threads_started":1}
W20260811 11:56:04.474671 16760 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:04.469187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:44959 (local address 127.6.143.190:46709)
0811 11:56:04.469349 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:04.469357 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:04.469591 (+ 234us) server_negotiation.cc:407] Connection header received
0811 11:56:04.469732 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:04.469734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:04.469796 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:04.469884 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:04.470383 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:04.470914 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:04.471698 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:04.471950 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:04.472026 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:04.474526 (+ 2500us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.130:44959: BlockingRecv error: recv got EOF from 127.6.143.130:44959 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 11:56:04 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:04 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:04.664707 16762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:04.658926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.131:38301)
0811 11:56:04.659208 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:04.659222 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:04.659320 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:04.659579 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:04.659581 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:04.659782 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:04.659892 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:04.659897 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:04.660729 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:04.660731 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:04.661484 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:04.661495 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:04.661888 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:04.661891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:04.661956 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:04.662600 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:04.662612 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:04.664531 (+ 1919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":109,"threads_started":1}
W20260811 11:56:04.664762 16760 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:04.659010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:38301 (local address 127.6.143.190:46709)
0811 11:56:04.659171 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:04.659175 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:04.659301 (+ 126us) server_negotiation.cc:407] Connection header received
0811 11:56:04.659406 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:04.659409 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:04.659456 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:04.659524 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:04.660089 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:04.660628 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:04.661587 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:04.661794 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:04.661876 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:04.664649 (+ 2773us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.131:38301: BlockingRecv error: recv got EOF from 127.6.143.131:38301 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 11 11:56:05 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:05 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:05.271653 16765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:05.266473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.129:42695)
0811 11:56:05.266757 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:05.266771 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:05.266873 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:05.267118 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:05.267120 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:05.267301 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:05.267407 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:05.267411 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:05.268192 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:05.268195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:05.268751 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:05.268759 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:05.269164 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:05.269167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:05.269228 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:05.269736 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:05.269747 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:05.271520 (+ 1773us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":90,"threads_started":1}
W20260811 11:56:05.271826 16766 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:05.266526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:42695 (local address 127.6.143.190:46709)
0811 11:56:05.266766 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:05.266769 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:05.266845 (+ 76us) server_negotiation.cc:407] Connection header received
0811 11:56:05.266954 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:05.266957 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:05.267000 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:05.267076 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:05.267504 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:05.268093 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:05.268847 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:05.269070 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:05.269146 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:05.271653 (+ 2507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.129:42695: BlockingRecv error: recv got EOF from 127.6.143.129:42695 (error 108)
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":63,"threads_started":1}
Aug 11 11:56:05 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:05 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:05.480661 16767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:05.475320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.130:35969)
0811 11:56:05.475616 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:05.475629 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:05.475731 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:05.476020 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:05.476022 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:05.476205 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:05.476305 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:05.476310 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:05.477021 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:05.477023 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:05.477593 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:05.477601 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:05.478006 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:05.478009 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:05.478067 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:05.478615 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:05.478628 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:05.480545 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":89,"threads_started":1}
W20260811 11:56:05.480937 16766 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:05.475407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:35969 (local address 127.6.143.190:46709)
0811 11:56:05.475542 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:05.475545 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:05.475694 (+ 149us) server_negotiation.cc:407] Connection header received
0811 11:56:05.475809 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:05.475811 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:05.475865 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:05.475988 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:05.476404 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:05.476927 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:05.477689 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:05.477910 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:05.477987 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:05.480678 (+ 2691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.130:35969: BlockingRecv error: recv got EOF from 127.6.143.130:35969 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 11:56:05 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:05 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:05.671799 16768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:05.665643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.131:34741)
0811 11:56:05.665991 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:05.666007 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:05.666156 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:05.666462 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:05.666465 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:05.666703 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:05.666850 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:05.666858 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:05.667626 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:05.667631 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:05.668288 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:05.668298 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:05.668764 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:05.668768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:05.668855 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:05.669412 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:05.669424 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:05.671603 (+ 2179us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":83,"threads_started":1}
W20260811 11:56:05.671835 16766 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:05.665823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:34741 (local address 127.6.143.190:46709)
0811 11:56:05.665965 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:05.665969 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:05.666092 (+ 123us) server_negotiation.cc:407] Connection header received
0811 11:56:05.666261 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:05.666267 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:05.666327 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:05.666421 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:05.666982 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:05.667511 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:05.668417 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:05.668641 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:05.668751 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:05.671729 (+ 2978us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.131:34741: BlockingRecv error: recv got EOF from 127.6.143.131:34741 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 11:56:06 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:06 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:06.278159 16769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:06.272557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.129:39823)
0811 11:56:06.272954 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:06.272967 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:06.273106 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:06.273442 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:06.273446 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:06.273719 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:06.273823 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:06.273829 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:06.274596 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:06.274598 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:06.275147 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:06.275154 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:06.275558 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:06.275560 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:06.275620 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:06.276144 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:06.276156 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:06.278037 (+ 1881us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":118,"threads_started":1}
W20260811 11:56:06.278348 16770 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:06.272562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:39823 (local address 127.6.143.190:46709)
0811 11:56:06.272874 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:06.272879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:06.273068 (+ 189us) server_negotiation.cc:407] Connection header received
0811 11:56:06.273214 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:06.273217 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:06.273278 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:06.273386 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:06.273967 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:06.274500 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:06.275250 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:06.275464 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:06.275539 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:06.278191 (+ 2652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.129:39823: BlockingRecv error: recv got EOF from 127.6.143.129:39823 (error 108)
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":86,"threads_started":1}
Aug 11 11:56:06 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:06 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:06.486948 16771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:06.481435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.130:44117)
0811 11:56:06.481758 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:06.481773 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:06.481911 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:06.482193 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:06.482196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:06.482419 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:06.482518 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:06.482524 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:06.483299 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:06.483304 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:06.483907 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:06.483916 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:06.484387 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:06.484391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:06.484478 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:06.484964 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:06.484976 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:06.486752 (+ 1776us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":103,"threads_started":1}
W20260811 11:56:06.487030 16770 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:06.481502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:44117 (local address 127.6.143.190:46709)
0811 11:56:06.481698 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:06.481705 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:06.481863 (+ 158us) server_negotiation.cc:407] Connection header received
0811 11:56:06.482021 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:06.482024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:06.482079 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:06.482164 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:06.482642 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:06.483183 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:06.484030 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:06.484264 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:06.484379 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:06.486908 (+ 2529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.130:44117: BlockingRecv error: recv got EOF from 127.6.143.130:44117 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Aug 11 11:56:06 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:06 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:06.678498 16772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:06.672912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.131:46821)
0811 11:56:06.673203 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:06.673217 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:06.673321 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:06.673569 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:06.673572 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:06.673766 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:06.673863 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:06.673868 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:06.674696 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:06.674701 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:06.675449 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:06.675460 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:06.675948 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:06.675952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:06.676041 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:06.676535 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:06.676547 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:06.678329 (+ 1782us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":110,"threads_started":1}
W20260811 11:56:06.678586 16770 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:06.672985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:46821 (local address 127.6.143.190:46709)
0811 11:56:06.673135 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:06.673140 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:06.673291 (+ 151us) server_negotiation.cc:407] Connection header received
0811 11:56:06.673413 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:06.673416 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:06.673463 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:06.673533 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:06.673972 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:06.674578 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:06.675573 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:06.675798 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:06.675946 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:06.678465 (+ 2519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.131:46821: BlockingRecv error: recv got EOF from 127.6.143.131:46821 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 11:56:07 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:07 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:07.284601 16773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:07.279045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.129:39535)
0811 11:56:07.279355 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:07.279368 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:07.279473 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:07.279737 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:07.279739 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:07.279977 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:07.280077 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:07.280083 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:07.280835 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:07.280838 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:07.281372 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:07.281379 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:07.281872 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:07.281876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:07.281957 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:07.282540 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:07.282554 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:07.284483 (+ 1929us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":78,"threads_started":1}
W20260811 11:56:07.284837 16774 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:07.279100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:39535 (local address 127.6.143.190:46709)
0811 11:56:07.279416 (+ 316us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:07.279420 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:07.279433 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:56:07.279575 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:07.279577 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:07.279619 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:07.279721 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:07.280175 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:07.280739 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:07.281491 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:07.281747 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:07.281849 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:07.284629 (+ 2780us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.129:39535: BlockingRecv error: recv got EOF from 127.6.143.129:39535 (error 108)
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":129,"threads_started":1}
Aug 11 11:56:07 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:07 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:07.497421 16775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:07.487854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.130:35171)
0811 11:56:07.488173 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:07.488190 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:07.488322 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:07.488656 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:07.488660 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:07.488897 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:07.489052 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:07.489062 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:07.489889 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:07.489892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:07.490434 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:07.490442 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:07.490848 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:07.490850 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:07.490951 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:07.495366 (+ 4415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:07.495379 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:07.497247 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":81,"threads_started":1}
W20260811 11:56:07.497623 16774 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:07.487948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:35171 (local address 127.6.143.190:46709)
0811 11:56:07.488230 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:07.488234 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:07.488252 (+ 18us) server_negotiation.cc:407] Connection header received
0811 11:56:07.488463 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:07.488466 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:07.488523 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:07.488613 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:07.489178 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:07.489784 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:07.490534 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:07.490749 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:07.490829 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:07.497374 (+ 6545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.130:35171: BlockingRecv error: recv got EOF from 127.6.143.130:35171 (error 108)
Metrics: {"server-negotiator.queue_time_us":184}
Aug 11 11:56:07 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:07 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:07.685765 16776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:07.679516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.131:50967)
0811 11:56:07.679879 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:07.679953 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:07.680085 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:07.680383 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:07.680386 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:07.680604 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:07.680699 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:07.680705 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:07.681475 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:07.681481 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:07.682049 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:07.682057 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:07.682526 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:07.682529 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:07.682617 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:07.683239 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:07.683251 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:07.685654 (+ 2403us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":105,"threads_started":1}
W20260811 11:56:07.685951 16774 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:07.679651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:50967 (local address 127.6.143.190:46709)
0811 11:56:07.679816 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:07.679820 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:07.680034 (+ 214us) server_negotiation.cc:407] Connection header received
0811 11:56:07.680206 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:07.680209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:07.680264 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:07.680351 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:07.680823 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:07.681349 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:07.682170 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:07.682406 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:07.682521 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:07.685781 (+ 3260us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.131:50967: BlockingRecv error: recv got EOF from 127.6.143.131:50967 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 11:56:08 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:08 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:08.291147 16777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:08.285487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.129:44397)
0811 11:56:08.285756 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:08.285769 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:08.285848 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:08.286119 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:08.286121 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:08.286347 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:08.286494 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:08.286506 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:08.287313 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:08.287316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:08.287944 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:08.287952 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:08.288492 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:08.288494 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:08.288557 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:08.289056 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:08.289068 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:08.291015 (+ 1947us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":96,"threads_started":1}
W20260811 11:56:08.291347 16778 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:08.285578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:44397 (local address 127.6.143.190:46709)
0811 11:56:08.285835 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:08.285838 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:08.285851 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:56:08.285958 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:08.285960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:08.286001 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:08.286081 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:08.286607 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:08.287197 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:08.288074 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:08.288377 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:08.288473 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:08.291195 (+ 2722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.129:44397: BlockingRecv error: recv got EOF from 127.6.143.129:44397 (error 108)
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":101,"threads_started":1}
Aug 11 11:56:08 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:08 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:08.504143 16779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:08.498322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.130:33449)
0811 11:56:08.498638 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:08.498652 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:08.498738 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:08.498998 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:08.499000 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:08.499188 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:08.499281 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:08.499287 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:08.500110 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:08.500113 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:08.500963 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:08.500973 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:08.501517 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:08.501521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:08.501627 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:08.502193 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:08.502210 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:08.504029 (+ 1819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":130,"threads_started":1}
W20260811 11:56:08.504281 16778 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:08.498538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:33449 (local address 127.6.143.190:46709)
0811 11:56:08.498701 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:08.498704 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:08.498719 (+ 15us) server_negotiation.cc:407] Connection header received
0811 11:56:08.498824 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:08.498826 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:08.498883 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:08.498957 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:08.499382 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:08.500002 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:08.501082 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:08.501385 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:08.501478 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:08.504163 (+ 2685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.130:33449: BlockingRecv error: recv got EOF from 127.6.143.130:33449 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 11:56:08 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:08 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:08.691892 16780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:08.686548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.131:36369)
0811 11:56:08.686845 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:08.686865 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:08.686958 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:08.687215 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:08.687217 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:08.687453 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:08.687555 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:08.687561 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:08.688315 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:08.688318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:08.688881 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:08.688890 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:08.689287 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:08.689290 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:08.689351 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:08.689833 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:08.689844 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:08.691742 (+ 1898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":97,"threads_started":1}
W20260811 11:56:08.692083 16778 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:08.686595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:36369 (local address 127.6.143.190:46709)
0811 11:56:08.686738 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:08.686741 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:08.686931 (+ 190us) server_negotiation.cc:407] Connection header received
0811 11:56:08.687039 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:08.687041 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:08.687087 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:08.687157 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:08.687662 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:08.688209 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:08.688981 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:08.689189 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:08.689266 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:08.691937 (+ 2671us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.131:36369: BlockingRecv error: recv got EOF from 127.6.143.131:36369 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 11:56:09 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:09 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:09.297757 16781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:09.292005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.129:60837)
0811 11:56:09.292293 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:09.292307 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:09.292403 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:09.292738 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:09.292741 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:09.293009 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:09.293119 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:09.293148 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:09.294085 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:09.294088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:09.294633 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:09.294641 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:09.295052 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:09.295055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:09.295114 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:09.295684 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:09.295696 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:09.297585 (+ 1889us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":76,"threads_started":1}
W20260811 11:56:09.297829 16782 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:09.292053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:60837 (local address 127.6.143.190:46709)
0811 11:56:09.292345 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:09.292349 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:09.292362 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:56:09.292521 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:09.292529 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:09.292599 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:09.292696 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:09.293262 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:09.293984 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:09.294733 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:09.294953 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:09.295032 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:09.297706 (+ 2674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.129:60837: BlockingRecv error: recv got EOF from 127.6.143.129:60837 (error 108)
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":121,"threads_started":1}
Aug 11 11:56:09 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:09 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:09.510728 16783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:09.505089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.130:59691)
0811 11:56:09.505382 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:09.505394 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:09.505536 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:09.505818 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:09.505821 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:09.506038 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:09.506132 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:09.506138 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:09.506913 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:09.506919 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:09.507561 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:09.507571 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:09.508096 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:09.508100 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:09.508188 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:09.508709 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:09.508722 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:09.510563 (+ 1841us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":91,"threads_started":1}
W20260811 11:56:09.510821 16782 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:09.505147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:59691 (local address 127.6.143.190:46709)
0811 11:56:09.505320 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:09.505326 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:09.505472 (+ 146us) server_negotiation.cc:407] Connection header received
0811 11:56:09.505651 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:09.505655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:09.505700 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:09.505787 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:09.506253 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:09.506801 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:09.507686 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:09.507979 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:09.508091 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:09.510700 (+ 2609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.130:59691: BlockingRecv error: recv got EOF from 127.6.143.130:59691 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 11:56:09 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:09 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:09.698364 16784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:09.692699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.131:58197)
0811 11:56:09.693015 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:09.693028 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:09.693162 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:09.693458 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:09.693461 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:09.693698 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:09.693790 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:09.693797 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:09.694567 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:09.694572 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:09.695161 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:09.695170 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:09.695708 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:09.695712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:09.695799 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:09.696446 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:09.696460 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:09.698249 (+ 1789us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":109,"threads_started":1}
W20260811 11:56:09.698506 16782 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:09.692756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:58197 (local address 127.6.143.190:46709)
0811 11:56:09.692920 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:09.692924 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:09.693107 (+ 183us) server_negotiation.cc:407] Connection header received
0811 11:56:09.693275 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:09.693278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:09.693332 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:09.693434 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:09.693920 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:09.694453 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:09.695288 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:09.695590 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:09.695673 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:09.698382 (+ 2709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.131:58197: BlockingRecv error: recv got EOF from 127.6.143.131:58197 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 11:56:10 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:10 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:10.303818 16785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:10.298630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.129:40027)
0811 11:56:10.298998 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:10.299011 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:10.299103 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:10.299334 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:10.299336 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:10.299519 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:10.299614 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:10.299619 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:10.300505 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:10.300508 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:10.301048 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:10.301055 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:10.301475 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:10.301478 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:10.301536 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:10.301998 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:10.302009 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:10.303702 (+ 1693us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":109,"threads_started":1}
W20260811 11:56:10.304028 16786 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:10.298799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:40027 (local address 127.6.143.190:46709)
0811 11:56:10.299066 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:10.299070 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:10.299084 (+ 14us) server_negotiation.cc:407] Connection header received
0811 11:56:10.299196 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:10.299198 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:10.299242 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:10.299322 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:10.299714 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:10.300396 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:10.301149 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:10.301366 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:10.301456 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:10.303833 (+ 2377us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.129:40027: BlockingRecv error: recv got EOF from 127.6.143.129:40027 (error 108)
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":120,"threads_started":1}
Aug 11 11:56:10 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:10 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:10.517019 16787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:10.511422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.130:35845)
0811 11:56:10.511738 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:10.511752 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:10.511852 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:10.512113 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:10.512115 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:10.512308 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:10.512417 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:10.512423 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:10.513144 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:10.513146 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:10.513852 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:10.513867 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:10.514385 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:10.514390 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:10.514489 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:10.515085 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:10.515097 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:10.516903 (+ 1806us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":99,"threads_started":1}
W20260811 11:56:10.517158 16786 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:10.511626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:35845 (local address 127.6.143.190:46709)
0811 11:56:10.511772 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:10.511776 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:10.511804 (+ 28us) server_negotiation.cc:407] Connection header received
0811 11:56:10.511956 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:10.511958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:10.512004 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:10.512076 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:10.512524 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:10.513051 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:10.513999 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:10.514264 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:10.514347 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:10.517039 (+ 2692us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.130:35845: BlockingRecv error: recv got EOF from 127.6.143.130:35845 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 11:56:10 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:10 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:10.704981 16789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:10.699309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.131:38279)
0811 11:56:10.699606 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:10.699619 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:10.699717 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:10.699994 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:10.699999 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:10.700191 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:10.700289 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:10.700295 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:10.701074 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:10.701080 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:10.701642 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:10.701652 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:10.702068 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:10.702071 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:10.702135 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:10.702588 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:10.702599 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:10.704865 (+ 2266us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":78,"threads_started":1}
W20260811 11:56:10.705248 16786 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:10.699364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:38279 (local address 127.6.143.190:46709)
0811 11:56:10.699489 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:10.699492 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:10.699705 (+ 213us) server_negotiation.cc:407] Connection header received
0811 11:56:10.699787 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:10.699789 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:10.699834 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:10.699972 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:10.700394 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:10.700967 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:10.701745 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:10.701968 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:10.702046 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:10.705003 (+ 2957us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.131:38279: BlockingRecv error: recv got EOF from 127.6.143.131:38279 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 11 11:56:11 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:11 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:11.310387 16790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:11.304598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.129:45213)
0811 11:56:11.305002 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:11.305016 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:11.305115 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:11.305377 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:11.305379 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:11.305566 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:11.305665 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:11.305671 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:11.306408 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:11.306411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:11.306986 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:11.306995 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:11.307465 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:11.307470 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:11.307588 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:11.308274 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:11.308294 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:11.310213 (+ 1919us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":211,"threads_started":1}
W20260811 11:56:11.310461 16791 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:11.304858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:45213 (local address 127.6.143.190:46709)
0811 11:56:11.305088 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:11.305092 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:11.305105 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:56:11.305212 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:11.305215 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:11.305259 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:11.305339 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:11.305770 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:11.306308 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:11.307087 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:11.307332 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:11.307420 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:11.310337 (+ 2917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.129:45213: BlockingRecv error: recv got EOF from 127.6.143.129:45213 (error 108)
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":67,"threads_started":1}
Aug 11 11:56:11 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:11 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:11.523061 16792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:11.517807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.130:32879)
0811 11:56:11.518151 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:11.518164 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:11.518261 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:11.518473 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:11.518475 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:11.518649 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:11.518745 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:11.518750 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:11.519655 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:11.519657 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:11.520233 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:11.520241 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:11.520661 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:11.520664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:11.520724 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:11.521193 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:11.521204 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:11.522947 (+ 1743us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":126,"threads_started":1}
W20260811 11:56:11.523195 16791 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:11.517850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:32879 (local address 127.6.143.190:46709)
0811 11:56:11.518004 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:11.518009 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:11.518238 (+ 229us) server_negotiation.cc:407] Connection header received
0811 11:56:11.518342 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:11.518344 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:11.518399 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:11.518460 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:11.518841 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:11.519535 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:11.520335 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:11.520562 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:11.520642 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:11.523076 (+ 2434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.130:32879: BlockingRecv error: recv got EOF from 127.6.143.130:32879 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 11:56:11 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:11 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:11.711143 16794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:11.705838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.131:54703)
0811 11:56:11.706104 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:11.706116 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:11.706212 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:11.706447 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:11.706449 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:11.706649 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:11.706745 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:11.706750 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:11.707494 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:11.707496 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:11.708100 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:11.708108 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:11.708521 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:11.708524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:11.708588 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:11.709066 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:11.709077 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:11.710997 (+ 1920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":98,"threads_started":1}
W20260811 11:56:11.711310 16791 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:11.705895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:54703 (local address 127.6.143.190:46709)
0811 11:56:11.706021 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:11.706024 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:11.706179 (+ 155us) server_negotiation.cc:407] Connection header received
0811 11:56:11.706279 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:11.706281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:11.706324 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:11.706402 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:11.706844 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:11.707366 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:11.708199 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:11.708422 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:11.708502 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:11.711159 (+ 2657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.131:54703: BlockingRecv error: recv got EOF from 127.6.143.131:54703 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 11 11:56:12 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:12 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:12.316746 16795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:12.311124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.129:53019)
0811 11:56:12.311427 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:12.311441 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:12.311537 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:12.311776 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:12.311778 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:12.311984 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:12.312084 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:12.312089 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:12.312798 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:12.312801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:12.313334 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:12.313341 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:12.313756 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:12.313759 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:12.313821 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:12.314300 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:12.314311 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:12.316613 (+ 2302us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":75,"threads_started":1}
W20260811 11:56:12.316888 16796 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:12.311214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:53019 (local address 127.6.143.190:46709)
0811 11:56:12.311486 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:12.311489 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:12.311503 (+ 14us) server_negotiation.cc:407] Connection header received
0811 11:56:12.311619 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:12.311621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:12.311663 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:12.311737 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:12.312186 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:12.312708 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:12.313433 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:12.313649 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:12.313736 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:12.316741 (+ 3005us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.129:53019: BlockingRecv error: recv got EOF from 127.6.143.129:53019 (error 108)
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":82,"threads_started":1}
Aug 11 11:56:12 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:12 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:12.532697 16797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:12.524090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.130:55343)
0811 11:56:12.524371 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:12.524384 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:12.524483 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:12.524734 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:12.524737 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:12.524929 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:12.525045 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:12.525050 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:12.525799 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:12.525806 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:12.526491 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:12.526503 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:12.526976 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:12.526979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:12.527088 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:12.530552 (+ 3464us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:12.530565 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:12.532582 (+ 2017us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":84,"threads_started":1}
W20260811 11:56:12.532841 16796 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:12.524156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:55343 (local address 127.6.143.190:46709)
0811 11:56:12.524290 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:12.524293 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:12.524447 (+ 154us) server_negotiation.cc:407] Connection header received
0811 11:56:12.524560 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:12.524566 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:12.524627 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:12.524698 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:12.525138 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:12.525685 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:12.526625 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:12.526875 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:12.526989 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:12.532722 (+ 5733us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.130:55343: BlockingRecv error: recv got EOF from 127.6.143.130:55343 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 11:56:12 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:12 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:12.717131 16798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:12.711855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.131:60907)
0811 11:56:12.712156 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:12.712171 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:12.712296 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:12.712530 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:12.712532 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:12.712715 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:12.712811 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:12.712816 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:12.713500 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:12.713502 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:12.714073 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:12.714081 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:12.714489 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:12.714491 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:12.714552 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:12.715049 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:12.715061 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:12.716998 (+ 1937us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":142,"threads_started":1}
W20260811 11:56:12.717283 16796 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:12.711977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:60907 (local address 127.6.143.190:46709)
0811 11:56:12.712108 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:12.712112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:12.712249 (+ 137us) server_negotiation.cc:407] Connection header received
0811 11:56:12.712368 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:12.712370 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:12.712413 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:12.712490 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:12.712908 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:12.713412 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:12.714173 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:12.714394 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:12.714470 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:12.717129 (+ 2659us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.131:60907: BlockingRecv error: recv got EOF from 127.6.143.131:60907 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 11 11:56:13 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:13 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:13.322978 16799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:13.317724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.129:60271)
0811 11:56:13.318006 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:13.318020 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:13.318107 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:13.318350 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:13.318352 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:13.318525 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:13.318624 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:13.318628 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:13.319342 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:13.319345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:13.319924 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:13.319933 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:13.320357 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:13.320359 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:13.320420 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:13.320990 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:13.321003 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:13.322865 (+ 1862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":91,"threads_started":1}
W20260811 11:56:13.323124 16801 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:13.317787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:60271 (local address 127.6.143.190:46709)
0811 11:56:13.318041 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:13.318044 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:13.318060 (+ 16us) server_negotiation.cc:407] Connection header received
0811 11:56:13.318190 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:13.318192 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:13.318234 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:13.318310 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:13.318717 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:13.319247 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:13.320026 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:13.320257 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:13.320338 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:13.322988 (+ 2650us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.129:60271: BlockingRecv error: recv got EOF from 127.6.143.129:60271 (error 108)
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":98,"threads_started":1}
Aug 11 11:56:13 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:13 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:13.539008 16802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:13.533521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.130:42027)
0811 11:56:13.533803 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:13.533817 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:13.533950 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:13.534199 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:13.534202 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:13.534420 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:13.534514 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:13.534520 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:13.535307 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:13.535312 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:13.535940 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:13.535952 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:13.536430 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:13.536433 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:13.536521 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:13.537029 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:13.537042 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:13.538895 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":88,"threads_started":1}
W20260811 11:56:13.539155 16801 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:13.533575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:42027 (local address 127.6.143.190:46709)
0811 11:56:13.533747 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:13.533753 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:13.533895 (+ 142us) server_negotiation.cc:407] Connection header received
0811 11:56:13.534037 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:13.534040 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:13.534093 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:13.534181 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:13.534630 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:13.535175 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:13.536070 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:13.536304 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:13.536418 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:13.539036 (+ 2618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.130:42027: BlockingRecv error: recv got EOF from 127.6.143.130:42027 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 11:56:13 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:13 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:13.723681 16803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:13.718053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.131:59253)
0811 11:56:13.718374 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:13.718392 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:13.718534 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:13.718822 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:13.718826 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:13.719053 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:13.719143 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:13.719149 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:13.719922 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:13.719929 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:13.720484 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:13.720492 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:13.720999 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:13.721003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:13.721091 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:13.721638 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:13.721658 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:13.723513 (+ 1855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":90,"threads_started":1}
W20260811 11:56:13.723747 16801 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:13.718205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:59253 (local address 127.6.143.190:46709)
0811 11:56:13.718355 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:13.718360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:13.718483 (+ 123us) server_negotiation.cc:407] Connection header received
0811 11:56:13.718646 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:13.718649 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:13.718701 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:13.718798 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:13.719267 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:13.719782 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:13.720607 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:13.720840 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:13.720996 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:13.723637 (+ 2641us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.131:59253: BlockingRecv error: recv got EOF from 127.6.143.131:59253 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 11:56:14 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:14 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:14.329021 16804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:14.323749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.129:37623)
0811 11:56:14.324076 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:14.324090 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:14.324181 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:14.324418 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:14.324420 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:14.324597 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:14.324692 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:14.324697 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:14.325424 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:14.325426 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:14.325970 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:14.325977 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:14.326390 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:14.326393 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:14.326462 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:14.327066 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:14.327079 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:14.328904 (+ 1825us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":91,"threads_started":1}
W20260811 11:56:14.329226 16805 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:14.323827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:37623 (local address 127.6.143.190:46709)
0811 11:56:14.324148 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:14.324152 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:14.324166 (+ 14us) server_negotiation.cc:407] Connection header received
0811 11:56:14.324273 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:14.324275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:14.324318 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:14.324398 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:14.324791 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:14.325322 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:14.326071 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:14.326296 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:14.326371 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:14.329047 (+ 2676us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.129:37623: BlockingRecv error: recv got EOF from 127.6.143.129:37623 (error 108)
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":129,"threads_started":1}
Aug 11 11:56:14 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:14 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:14.545503 16806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:14.540235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.130:36093)
0811 11:56:14.540553 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:14.540569 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:14.540684 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:14.541013 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:14.541016 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:14.541229 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:14.541331 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:14.541337 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:14.542052 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:14.542054 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:14.542586 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:14.542593 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:14.542999 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:14.543002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:14.543061 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:14.543562 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:14.543573 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:14.545364 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":116,"threads_started":1}
W20260811 11:56:14.545639 16805 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:14.540330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:36093 (local address 127.6.143.190:46709)
0811 11:56:14.540487 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:14.540492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:14.540659 (+ 167us) server_negotiation.cc:407] Connection header received
0811 11:56:14.540795 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:14.540801 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:14.540867 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:14.540968 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:14.541436 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:14.541957 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:14.542688 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:14.542903 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:14.542978 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:14.545520 (+ 2542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.130:36093: BlockingRecv error: recv got EOF from 127.6.143.130:36093 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 11 11:56:14 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:14 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:14.729578 16808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:14.724437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.131:42241)
0811 11:56:14.724735 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:14.724748 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:14.724851 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:14.725084 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:14.725086 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:14.725264 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:14.725380 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:14.725386 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:14.726121 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:14.726123 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:14.726677 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:14.726685 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:14.727094 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:14.727096 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:14.727156 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:14.727721 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:14.727733 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:14.729468 (+ 1735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":112,"threads_started":1}
W20260811 11:56:14.729713 16805 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:14.724523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:42241 (local address 127.6.143.190:46709)
0811 11:56:14.724671 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:14.724676 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:14.724816 (+ 140us) server_negotiation.cc:407] Connection header received
0811 11:56:14.724932 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:14.724935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:14.724981 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:14.725048 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:14.725475 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:14.725985 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:14.726775 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:14.726996 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:14.727074 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:14.729595 (+ 2521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.131:42241: BlockingRecv error: recv got EOF from 127.6.143.131:42241 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 11:56:15 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:15 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:15.334834 16809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:15.329794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.129:47991)
0811 11:56:15.330081 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:15.330095 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:15.330185 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:15.330431 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:15.330434 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:15.330611 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:15.330707 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:15.330712 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:15.331410 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:15.331413 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:15.332004 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:15.332011 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:15.332416 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:15.332418 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:15.332475 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:15.332951 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:15.332961 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:15.334723 (+ 1762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":95,"threads_started":1}
W20260811 11:56:15.335003 16810 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:15.329877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:47991 (local address 127.6.143.190:46709)
0811 11:56:15.330150 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:15.330153 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:15.330167 (+ 14us) server_negotiation.cc:407] Connection header received
0811 11:56:15.330273 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:15.330275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:15.330317 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:15.330389 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:15.330799 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:15.331321 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:15.332114 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:15.332328 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:15.332398 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:15.334853 (+ 2455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.129:47991: BlockingRecv error: recv got EOF from 127.6.143.129:47991 (error 108)
Metrics: {"server-negotiator.queue_time_us":205,"thread_start_us":64,"threads_started":1}
Aug 11 11:56:15 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:15 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:15.551928 16811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:15.546388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.130:42019)
0811 11:56:15.546681 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:15.546704 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:15.546847 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:15.547149 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:15.547152 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:15.547405 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:15.547548 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:15.547557 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:15.548405 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:15.548407 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:15.548964 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:15.548972 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:15.549379 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:15.549382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:15.549443 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:15.549934 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:15.549945 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:15.551780 (+ 1835us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":112,"threads_started":1}
W20260811 11:56:15.552212 16810 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:15.546507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:42019 (local address 127.6.143.190:46709)
0811 11:56:15.546653 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:15.546658 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:15.546785 (+ 127us) server_negotiation.cc:407] Connection header received
0811 11:56:15.546962 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:15.546966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:15.547016 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:15.547118 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:15.547663 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:15.548302 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:15.549067 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:15.549282 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:15.549357 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:15.551987 (+ 2630us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.130:42019: BlockingRecv error: recv got EOF from 127.6.143.130:42019 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 11 11:56:15 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:15 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:15.736236 16812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:15.730424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.131:52121)
0811 11:56:15.730885 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:15.730897 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:15.731037 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:15.731314 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:15.731317 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:15.731553 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:15.731665 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:15.731671 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:15.732492 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:15.732497 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:15.733071 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:15.733081 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:15.733561 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:15.733565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:15.733662 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:15.734224 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:15.734236 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:15.736068 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":111,"threads_started":1}
W20260811 11:56:15.736286 16810 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:15.730629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:52121 (local address 127.6.143.190:46709)
0811 11:56:15.730773 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:15.730779 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:15.730973 (+ 194us) server_negotiation.cc:407] Connection header received
0811 11:56:15.731152 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:15.731155 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:15.731202 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:15.731284 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:15.731784 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:15.732381 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:15.733207 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:15.733437 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:15.733543 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:15.736182 (+ 2639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.131:52121: BlockingRecv error: recv got EOF from 127.6.143.131:52121 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 11 11:56:16 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:16 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:16.340840 16814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:16.335573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.129:60735)
0811 11:56:16.335843 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:16.335857 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:16.336008 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:16.336241 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:16.336243 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:16.336440 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:16.336541 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:16.336547 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:16.337277 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:16.337280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:16.337846 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:16.337854 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:16.338296 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:16.338298 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:16.338367 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:16.338872 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:16.338883 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:16.340716 (+ 1833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":94,"threads_started":1}
W20260811 11:56:16.341037 16815 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:16.335765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:60735 (local address 127.6.143.190:46709)
0811 11:56:16.336025 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:16.336029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:16.336042 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:56:16.336078 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:16.336080 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:16.336120 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:16.336203 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:16.336648 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:16.337180 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:16.337965 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:16.338194 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:16.338276 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:16.340852 (+ 2576us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.129:60735: BlockingRecv error: recv got EOF from 127.6.143.129:60735 (error 108)
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":87,"threads_started":1}
Aug 11 11:56:16 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:16 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:16.557749 16816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:16.552613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.130:35631)
0811 11:56:16.552924 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:16.552936 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:16.553029 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:16.553258 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:16.553260 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:16.553440 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:16.553533 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:16.553538 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:16.554253 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:16.554255 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:16.554785 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:16.554792 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:16.555194 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:16.555196 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:16.555257 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:16.555748 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:16.555760 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:16.557636 (+ 1876us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":80,"threads_started":1}
W20260811 11:56:16.558059 16815 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:16.552771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:35631 (local address 127.6.143.190:46709)
0811 11:56:16.552925 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:16.552929 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:16.552997 (+ 68us) server_negotiation.cc:407] Connection header received
0811 11:56:16.553105 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:16.553107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:16.553149 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:16.553223 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:16.553632 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:16.554159 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:16.554880 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:16.555097 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:16.555171 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:16.557828 (+ 2657us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.130:35631: BlockingRecv error: recv got EOF from 127.6.143.130:35631 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 11 11:56:16 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:16 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:16.742578 16817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:16.737051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.131:38937)
0811 11:56:16.737359 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:16.737372 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:16.737512 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:16.737801 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:16.737804 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:16.738030 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:16.738126 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:16.738132 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:16.738911 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:16.738915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:16.739493 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:16.739501 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:16.740031 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:16.740035 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:16.740125 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:16.740620 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:16.740632 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:16.742394 (+ 1762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":102,"threads_started":1}
W20260811 11:56:16.742645 16815 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:16.737133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:38937 (local address 127.6.143.190:46709)
0811 11:56:16.737299 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:16.737304 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:16.737449 (+ 145us) server_negotiation.cc:407] Connection header received
0811 11:56:16.737626 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:16.737635 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:16.737683 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:16.737771 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:16.738251 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:16.738786 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:16.739620 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:16.739847 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:16.740016 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:16.742529 (+ 2513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.131:38937: BlockingRecv error: recv got EOF from 127.6.143.131:38937 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 11:56:17 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:17 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:17.346963 16818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:17.341602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.129:46487)
0811 11:56:17.341895 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:17.341908 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:17.341999 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:17.342222 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:17.342225 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:17.342420 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:17.342514 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:17.342519 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:17.343259 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:17.343262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:17.343835 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:17.343842 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:17.344324 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:17.344327 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:17.344390 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:17.344881 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:17.344893 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:17.346828 (+ 1935us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":111,"threads_started":1}
W20260811 11:56:17.347111 16819 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:17.341645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:46487 (local address 127.6.143.190:46709)
0811 11:56:17.341948 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:17.341951 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:17.341965 (+ 14us) server_negotiation.cc:407] Connection header received
0811 11:56:17.342089 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:17.342091 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:17.342135 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:17.342212 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:17.342613 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:17.343165 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:17.343995 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:17.344222 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:17.344302 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:17.346976 (+ 2674us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.129:46487: BlockingRecv error: recv got EOF from 127.6.143.129:46487 (error 108)
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":88,"threads_started":1}
Aug 11 11:56:17 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:17 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:17.568728 16820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:17.558649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.130:53861)
0811 11:56:17.558978 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:17.558999 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:17.559141 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:17.559414 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:17.559417 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:17.559639 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:17.559752 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:17.559759 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:17.560667 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:17.560673 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:17.561320 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:17.561329 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:17.561787 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:17.561791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:17.561883 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:17.566658 (+ 4775us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:17.566671 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:17.568559 (+ 1888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":109,"threads_started":1}
W20260811 11:56:17.568830 16819 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:17.558742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:53861 (local address 127.6.143.190:46709)
0811 11:56:17.558944 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:17.558949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:17.559074 (+ 125us) server_negotiation.cc:407] Connection header received
0811 11:56:17.559251 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:17.559256 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:17.559306 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:17.559390 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:17.559914 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:17.560556 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:17.561443 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:17.561670 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:17.561784 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:17.568710 (+ 6926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.130:53861: BlockingRecv error: recv got EOF from 127.6.143.130:53861 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 11 11:56:17 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:17 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.131@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:17.749019 16821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:17.743488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.131:37629)
0811 11:56:17.743756 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:17.743769 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:17.743899 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:17.744237 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:17.744240 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:17.744521 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:17.744623 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:17.744629 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:17.745382 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:17.745385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:17.746006 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:17.746016 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:17.746453 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:17.746456 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:17.746519 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:17.747006 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:17.747017 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:17.748890 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":88,"threads_started":1}
W20260811 11:56:17.749410 16819 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:17.743564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:37629 (local address 127.6.143.190:46709)
0811 11:56:17.743745 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:17.743750 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:17.743841 (+ 91us) server_negotiation.cc:407] Connection header received
0811 11:56:17.744039 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:17.744041 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:17.744101 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:17.744196 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:17.744756 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:17.745285 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:17.746120 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:17.746344 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:17.746426 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:17.749292 (+ 2866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.131:37629: BlockingRecv error: recv got EOF from 127.6.143.131:37629 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 11:56:18 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:18 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.129@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:18.353497 16822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:18.347880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.129:54167)
0811 11:56:18.348244 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:18.348258 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:18.348349 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:18.348605 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:18.348607 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:18.348824 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:18.348922 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:18.348927 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:18.349639 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:18.349642 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:18.350223 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:18.350231 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:18.350737 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:18.350740 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:18.350811 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:18.351407 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:18.351419 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:18.353303 (+ 1884us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":71,"threads_started":1}
W20260811 11:56:18.353556 16823 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:18.348000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:54167 (local address 127.6.143.190:46709)
0811 11:56:18.348292 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:18.348296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:18.348308 (+ 12us) server_negotiation.cc:407] Connection header received
0811 11:56:18.348443 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:18.348445 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:18.348487 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:18.348563 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:18.349028 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:18.349545 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:18.350325 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:18.350543 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:18.350617 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:18.353428 (+ 2811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.129:54167: BlockingRecv error: recv got EOF from 127.6.143.129:54167 (error 108)
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":103,"threads_started":1}
Aug 11 11:56:18 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 11:56:18 dist-test-slave-zkpd krb5kdc[15615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.143.130@KRBTEST.COM for oryx/127.6.143.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 11:56:18.575181 16824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:18.569682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46709 (local address 127.6.143.130:36281)
0811 11:56:18.570004 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:18.570022 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:18.570158 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:18.570447 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:18.570450 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:18.570674 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:18.570766 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:18.570772 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:18.571548 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:18.571553 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:18.572133 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:18.572142 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:18.572625 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:18.572630 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:18.572720 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:18.573220 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:18.573232 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:18.575049 (+ 1817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.143.190:46709: Server oryx/127.6.143.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":88,"threads_started":1}
W20260811 11:56:18.575309 16823 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:18.569806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:36281 (local address 127.6.143.190:46709)
0811 11:56:18.569988 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:18.569993 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:18.570110 (+ 117us) server_negotiation.cc:407] Connection header received
0811 11:56:18.570270 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:18.570274 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:18.570328 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:18.570421 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:18.570900 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:18.571433 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:18.572265 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:18.572504 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:18.572622 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:18.575188 (+ 2566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.143.130:36281: BlockingRecv error: recv got EOF from 127.6.143.130:36281 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 11:56:18.647562 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 16292
I20260811 11:56:18.656350 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 16464
I20260811 11:56:18.664363 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 16611
I20260811 11:56:18.671422 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 16209
2026-08-11T11:56:18Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18069 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T11:56:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:56:18Z Disabled control of system clock
I20260811 11:56:18.706001 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:36315
--webserver_interface=127.6.143.190
--webserver_port=0
--builtin_ntp_servers=127.6.143.148:33815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:36315
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 11:56:18.821285 16828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:56:18.821528 16828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:56:18.821563 16828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:56:18.825449 16828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:56:18.825503 16828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:56:18.825519 16828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:56:18.825536 16828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:56:18.825551 16828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:56:18.830166 16828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:33815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:36315
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:36315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:56:18.831219 16828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:56:18.832602 16828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:56:18.841578 16834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:18.841599 16833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:18.841599 16836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:56:18.841655 16828 server_base.cc:1034] running on GCE node
I20260811 11:56:18.842478 16828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:56:18.843513 16828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:56:18.844708 16828 hybrid_clock.cc:648] HybridClock initialized: now 1786449378844694 us; error 40 us; skew 500 ppm
I20260811 11:56:18.846537 16828 webserver.cc:460] Webserver started at http://127.6.143.190:33509/ using document root <none> and password file <none>
I20260811 11:56:18.847141 16828 fs_manager.cc:359] Metadata directory not provided
I20260811 11:56:18.847236 16828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:56:18.847469 16828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:56:18.849609 16828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "0c77f8de36d64e9092c36e7b1956b6d8"
format_stamp: "Formatted at 2026-08-11 11:56:18 on dist-test-slave-zkpd"
server_key: "1d8c853991f6ef1a0a3f005ac2a4fd9d"
server_key_iv: "58710ea4ea6f893736659b53e71b80c7"
server_key_version: "encryptionkey@0"
I20260811 11:56:18.850178 16828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "0c77f8de36d64e9092c36e7b1956b6d8"
format_stamp: "Formatted at 2026-08-11 11:56:18 on dist-test-slave-zkpd"
server_key: "1d8c853991f6ef1a0a3f005ac2a4fd9d"
server_key_iv: "58710ea4ea6f893736659b53e71b80c7"
server_key_version: "encryptionkey@0"
I20260811 11:56:18.853885 16828 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260811 11:56:18.856359 16842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:18.857595 16828 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:56:18.857744 16828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "0c77f8de36d64e9092c36e7b1956b6d8"
format_stamp: "Formatted at 2026-08-11 11:56:18 on dist-test-slave-zkpd"
server_key: "1d8c853991f6ef1a0a3f005ac2a4fd9d"
server_key_iv: "58710ea4ea6f893736659b53e71b80c7"
server_key_version: "encryptionkey@0"
I20260811 11:56:18.857861 16828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:56:18.866981 16828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:56:18.867612 16828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:56:18.867805 16828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:56:18.874845 16828 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:36315
I20260811 11:56:18.874859 16902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:36315 every 8 connection(s)
I20260811 11:56:18.875761 16828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:56:18.878762 16903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:56:18.881871 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 16828
I20260811 11:56:18.881965 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:56:18.882251 6718 external_mini_cluster.cc:1428] Setting key 37a6af13bbdcc53020152a70e88ed7b7
I20260811 11:56:18.884759 16903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c77f8de36d64e9092c36e7b1956b6d8: Bootstrap starting.
I20260811 11:56:18.887279 16903 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0c77f8de36d64e9092c36e7b1956b6d8: Neither blocks nor log segments found. Creating new log.
I20260811 11:56:18.888252 16903 log.cc:824] T 00000000000000000000000000000000 P 0c77f8de36d64e9092c36e7b1956b6d8: Log is configured to *not* fsync() on all Append() calls
I20260811 11:56:18.890524 16903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0c77f8de36d64e9092c36e7b1956b6d8: No bootstrap required, opened a new log
I20260811 11:56:18.892153 16906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:18.883713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44160 (local address 127.6.143.190:36315)
0811 11:56:18.884367 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:18.884383 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:18.884422 (+ 39us) server_negotiation.cc:407] Connection header received
0811 11:56:18.885104 (+ 682us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:18.885123 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:18.885521 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:18.885795 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:18.886164 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:18.887302 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:18.888595 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:18.889270 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:18.889392 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:18.890262 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:56:18.890267 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:56:18.890280 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:56:18.890717 (+ 437us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:56:18.890775 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:56:18.890796 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:56:18.890876 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 11:56:18.891078 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 11:56:18.891375 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":174,"threads_started":1}
I20260811 11:56:18.894774 16903 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0c77f8de36d64e9092c36e7b1956b6d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c77f8de36d64e9092c36e7b1956b6d8" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 36315 } }
I20260811 11:56:18.894989 16903 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0c77f8de36d64e9092c36e7b1956b6d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:56:18.895064 16903 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0c77f8de36d64e9092c36e7b1956b6d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c77f8de36d64e9092c36e7b1956b6d8, State: Initialized, Role: FOLLOWER
I20260811 11:56:18.895480 16903 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0c77f8de36d64e9092c36e7b1956b6d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c77f8de36d64e9092c36e7b1956b6d8" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 36315 } }
I20260811 11:56:18.895601 16903 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0c77f8de36d64e9092c36e7b1956b6d8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:56:18.895686 16903 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0c77f8de36d64e9092c36e7b1956b6d8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:56:18.895814 16903 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0c77f8de36d64e9092c36e7b1956b6d8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:56:18.896733 16903 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0c77f8de36d64e9092c36e7b1956b6d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c77f8de36d64e9092c36e7b1956b6d8" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 36315 } }
I20260811 11:56:18.897081 16903 leader_election.cc:302] T 00000000000000000000000000000000 P 0c77f8de36d64e9092c36e7b1956b6d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0c77f8de36d64e9092c36e7b1956b6d8; no voters:
I20260811 11:56:18.897384 16903 leader_election.cc:288] T 00000000000000000000000000000000 P 0c77f8de36d64e9092c36e7b1956b6d8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:56:18.897504 16908 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0c77f8de36d64e9092c36e7b1956b6d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:56:18.897761 16908 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0c77f8de36d64e9092c36e7b1956b6d8 [term 1 LEADER]: Becoming Leader. State: Replica: 0c77f8de36d64e9092c36e7b1956b6d8, State: Running, Role: LEADER
I20260811 11:56:18.898146 16908 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0c77f8de36d64e9092c36e7b1956b6d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c77f8de36d64e9092c36e7b1956b6d8" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 36315 } }
I20260811 11:56:18.898742 16903 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0c77f8de36d64e9092c36e7b1956b6d8 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:56:18.900060 16909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c77f8de36d64e9092c36e7b1956b6d8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0c77f8de36d64e9092c36e7b1956b6d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c77f8de36d64e9092c36e7b1956b6d8" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 36315 } } }
I20260811 11:56:18.900215 16909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c77f8de36d64e9092c36e7b1956b6d8 [sys.catalog]: This master's current role is: LEADER
I20260811 11:56:18.900090 16910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0c77f8de36d64e9092c36e7b1956b6d8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0c77f8de36d64e9092c36e7b1956b6d8. Latest consensus state: current_term: 1 leader_uuid: "0c77f8de36d64e9092c36e7b1956b6d8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0c77f8de36d64e9092c36e7b1956b6d8" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 36315 } } }
I20260811 11:56:18.900332 16910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0c77f8de36d64e9092c36e7b1956b6d8 [sys.catalog]: This master's current role is: LEADER
I20260811 11:56:18.900689 16917 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:56:18.903954 16917 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:56:18.909718 16917 catalog_manager.cc:1347] Generated new cluster ID: 06aa288f7ade4fd699348e653076db33
I20260811 11:56:18.909790 16917 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:56:18.925475 16917 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:56:18.926692 16917 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:56:18.939406 16917 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0c77f8de36d64e9092c36e7b1956b6d8: Generated new TSK 0
I20260811 11:56:18.940344 16917 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:56:18.944209 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:0
--local_ip_for_outbound_sockets=127.6.143.129
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:36315
--builtin_ntp_servers=127.6.143.148:33815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 11:56:19.060853 16928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:56:19.061100 16928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:56:19.061136 16928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:56:19.065122 16928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:56:19.065171 16928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:56:19.065227 16928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:56:19.069768 16928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:33815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:36315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:56:19.070813 16928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:56:19.072173 16928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:56:19.081664 16934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:19.081708 16935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:19.081684 16938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:56:19.081907 16928 server_base.cc:1034] running on GCE node
I20260811 11:56:19.082374 16928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:56:19.082985 16928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:56:19.084141 16928 hybrid_clock.cc:648] HybridClock initialized: now 1786449379084108 us; error 45 us; skew 500 ppm
I20260811 11:56:19.085930 16928 webserver.cc:460] Webserver started at http://127.6.143.129:40123/ using document root <none> and password file <none>
I20260811 11:56:19.086516 16928 fs_manager.cc:359] Metadata directory not provided
I20260811 11:56:19.086580 16928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:56:19.086838 16928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:56:19.088963 16928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/data/instance:
uuid: "0be8352f98394a7f9b28f6dbc97e625d"
format_stamp: "Formatted at 2026-08-11 11:56:19 on dist-test-slave-zkpd"
server_key: "ac943a5465a0bf04920bb84643db24d7"
server_key_iv: "28a9b963854fac12ae38a13aefcc4b52"
server_key_version: "encryptionkey@0"
I20260811 11:56:19.089522 16928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance:
uuid: "0be8352f98394a7f9b28f6dbc97e625d"
format_stamp: "Formatted at 2026-08-11 11:56:19 on dist-test-slave-zkpd"
server_key: "ac943a5465a0bf04920bb84643db24d7"
server_key_iv: "28a9b963854fac12ae38a13aefcc4b52"
server_key_version: "encryptionkey@0"
I20260811 11:56:19.093151 16928 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 11:56:19.095577 16945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:19.096845 16928 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:56:19.097012 16928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "0be8352f98394a7f9b28f6dbc97e625d"
format_stamp: "Formatted at 2026-08-11 11:56:19 on dist-test-slave-zkpd"
server_key: "ac943a5465a0bf04920bb84643db24d7"
server_key_iv: "28a9b963854fac12ae38a13aefcc4b52"
server_key_version: "encryptionkey@0"
I20260811 11:56:19.097149 16928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:56:19.113742 16928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:56:19.114442 16928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:56:19.114643 16928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:56:19.115236 16928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:56:19.116279 16928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:56:19.116351 16928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:19.116453 16928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:56:19.116506 16928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:19.125610 16928 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:37525
I20260811 11:56:19.125631 17064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:37525 every 8 connection(s)
I20260811 11:56:19.126480 16928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:56:19.131644 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 16928
I20260811 11:56:19.131738 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance
I20260811 11:56:19.132069 6718 external_mini_cluster.cc:1428] Setting key 86be107e4f8a952eb821926c69f10efd
I20260811 11:56:19.133981 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:0
--local_ip_for_outbound_sockets=127.6.143.130
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:36315
--builtin_ntp_servers=127.6.143.148:33815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 11:56:19.136153 16906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:19.128393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:56853 (local address 127.6.143.190:36315)
0811 11:56:19.128551 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:19.128555 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:19.129542 (+ 987us) server_negotiation.cc:407] Connection header received
0811 11:56:19.130884 (+ 1342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:19.130886 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:19.130933 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:19.131007 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:19.131996 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:19.132703 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:19.133996 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:19.134328 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:19.134451 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:19.135635 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:56:19.135639 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:56:19.135642 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:56:19.135662 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:56:19.135683 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:56:19.135691 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:56:19.135755 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 11:56:19.135995 (+ 240us) server_negotiation.cc:299] Negotiation successful
0811 11:56:19.136047 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 11:56:19.137262 17067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:19.128758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:36315 (local address 127.6.143.129:56853)
0811 11:56:19.129402 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:19.129441 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:19.130698 (+ 1257us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:19.131177 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:19.131184 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:19.131265 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:19.131791 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:19.131818 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:19.132837 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:19.132841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:19.133866 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:19.133876 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:19.134490 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:19.134494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:19.134663 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:19.135306 (+ 643us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:56:19.135308 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:19.135348 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:56:19.135443 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:56:19.135781 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:56:19.135792 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:56:19.135801 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 11:56:19.135974 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 11:56:19.136278 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":111,"threads_started":1}
I20260811 11:56:19.138454 17065 heartbeater.cc:347] Connected to a master server at 127.6.143.190:36315
I20260811 11:56:19.138723 17065 heartbeater.cc:464] Registering TS with master...
I20260811 11:56:19.139382 17065 heartbeater.cc:511] Master 127.6.143.190:36315 requested a full tablet report, sending...
I20260811 11:56:19.141059 16861 ts_manager.cc:194] Registered new tserver with Master: 0be8352f98394a7f9b28f6dbc97e625d (127.6.143.129:37525)
I20260811 11:56:19.142256 16861 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.143.129:56853
W20260811 11:56:19.247931 17068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:56:19.248172 17068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:56:19.248260 17068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:56:19.252235 17068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:56:19.252295 17068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:56:19.252396 17068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:56:19.257130 17068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:33815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:36315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:56:19.258224 17068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:56:19.259653 17068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:56:19.269209 17076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:56:19.269258 17068 server_base.cc:1034] running on GCE node
W20260811 11:56:19.269276 17078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:19.269240 17075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:56:19.269924 17068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:56:19.270515 17068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:56:19.271675 17068 hybrid_clock.cc:648] HybridClock initialized: now 1786449379271658 us; error 27 us; skew 500 ppm
I20260811 11:56:19.277313 17068 webserver.cc:460] Webserver started at http://127.6.143.130:43413/ using document root <none> and password file <none>
I20260811 11:56:19.278066 17068 fs_manager.cc:359] Metadata directory not provided
I20260811 11:56:19.278160 17068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:56:19.278375 17068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:56:19.280580 17068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/data/instance:
uuid: "7d2c92d1130b48fcbeecbbf9b7d84d99"
format_stamp: "Formatted at 2026-08-11 11:56:19 on dist-test-slave-zkpd"
server_key: "de855bda1e9748243d1249c0ac4bb4c3"
server_key_iv: "c663c27a633f229324584dc2f616eedb"
server_key_version: "encryptionkey@0"
I20260811 11:56:19.281162 17068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance:
uuid: "7d2c92d1130b48fcbeecbbf9b7d84d99"
format_stamp: "Formatted at 2026-08-11 11:56:19 on dist-test-slave-zkpd"
server_key: "de855bda1e9748243d1249c0ac4bb4c3"
server_key_iv: "c663c27a633f229324584dc2f616eedb"
server_key_version: "encryptionkey@0"
I20260811 11:56:19.284884 17068 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 11:56:19.287359 17086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:19.288654 17068 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:56:19.288793 17068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "7d2c92d1130b48fcbeecbbf9b7d84d99"
format_stamp: "Formatted at 2026-08-11 11:56:19 on dist-test-slave-zkpd"
server_key: "de855bda1e9748243d1249c0ac4bb4c3"
server_key_iv: "c663c27a633f229324584dc2f616eedb"
server_key_version: "encryptionkey@0"
I20260811 11:56:19.288910 17068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:56:19.304141 17068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:56:19.304845 17068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:56:19.305049 17068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:56:19.305642 17068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:56:19.306602 17068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:56:19.306674 17068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:19.306902 17068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:56:19.306985 17068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:19.315747 17068 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:33517
I20260811 11:56:19.315770 17209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:33517 every 8 connection(s)
I20260811 11:56:19.316676 17068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
I20260811 11:56:19.321581 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 17068
I20260811 11:56:19.321699 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance
I20260811 11:56:19.321995 6718 external_mini_cluster.cc:1428] Setting key f4af71f034bd620e173863ea86619ee9
I20260811 11:56:19.323966 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:0
--local_ip_for_outbound_sockets=127.6.143.131
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:36315
--builtin_ntp_servers=127.6.143.148:33815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 11:56:19.326465 16906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:19.318567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:42991 (local address 127.6.143.190:36315)
0811 11:56:19.318704 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:19.318708 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:19.319796 (+ 1088us) server_negotiation.cc:407] Connection header received
0811 11:56:19.321346 (+ 1550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:19.321350 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:19.321409 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:19.321522 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:19.322558 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:19.323080 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:19.324380 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:19.324600 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:19.324713 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:19.325910 (+ 1197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:56:19.325916 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:56:19.325919 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:56:19.325936 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:56:19.325963 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:56:19.325976 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:56:19.326058 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 11:56:19.326205 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 11:56:19.326272 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 11:56:19.327293 17213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:19.318984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:36315 (local address 127.6.143.130:42991)
0811 11:56:19.319645 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:19.319686 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:19.321118 (+ 1432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:19.321712 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:19.321724 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:19.321859 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:19.322405 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:19.322420 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:19.323217 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:19.323220 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:19.324257 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:19.324267 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:19.324754 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:19.324758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:19.324938 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:19.325620 (+ 682us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:56:19.325624 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:19.325677 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:56:19.325753 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:56:19.326067 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:56:19.326074 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:56:19.326079 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:56:19.326175 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 11:56:19.326397 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":113,"threads_started":1}
I20260811 11:56:19.328382 17211 heartbeater.cc:347] Connected to a master server at 127.6.143.190:36315
I20260811 11:56:19.328636 17211 heartbeater.cc:464] Registering TS with master...
I20260811 11:56:19.329137 17211 heartbeater.cc:511] Master 127.6.143.190:36315 requested a full tablet report, sending...
I20260811 11:56:19.330168 16861 ts_manager.cc:194] Registered new tserver with Master: 7d2c92d1130b48fcbeecbbf9b7d84d99 (127.6.143.130:33517)
I20260811 11:56:19.330727 16861 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.143.130:42991
W20260811 11:56:19.451866 17214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:56:19.452178 17214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:56:19.452244 17214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:56:19.456238 17214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:56:19.456287 17214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:56:19.456377 17214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:56:19.463109 17214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:33815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:36315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:56:19.464437 17214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:56:19.466256 17214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:56:19.475857 17222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:19.475869 17220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:19.475849 17219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:56:19.476406 17214 server_base.cc:1034] running on GCE node
I20260811 11:56:19.476783 17214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:56:19.477430 17214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:56:19.478600 17214 hybrid_clock.cc:648] HybridClock initialized: now 1786449379478582 us; error 26 us; skew 500 ppm
I20260811 11:56:19.480554 17214 webserver.cc:460] Webserver started at http://127.6.143.131:44901/ using document root <none> and password file <none>
I20260811 11:56:19.481148 17214 fs_manager.cc:359] Metadata directory not provided
I20260811 11:56:19.481210 17214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:56:19.481462 17214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:56:19.483539 17214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/data/instance:
uuid: "9081b85b5df84f5cb812f5f1a5d859b1"
format_stamp: "Formatted at 2026-08-11 11:56:19 on dist-test-slave-zkpd"
server_key: "4e83b0630f0009ff33674bf75b977b18"
server_key_iv: "41876159452554a7e2f030dede62c9a3"
server_key_version: "encryptionkey@0"
I20260811 11:56:19.484133 17214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance:
uuid: "9081b85b5df84f5cb812f5f1a5d859b1"
format_stamp: "Formatted at 2026-08-11 11:56:19 on dist-test-slave-zkpd"
server_key: "4e83b0630f0009ff33674bf75b977b18"
server_key_iv: "41876159452554a7e2f030dede62c9a3"
server_key_version: "encryptionkey@0"
I20260811 11:56:19.487758 17214 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 11:56:19.490221 17229 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:19.491443 17214 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:56:19.491578 17214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "9081b85b5df84f5cb812f5f1a5d859b1"
format_stamp: "Formatted at 2026-08-11 11:56:19 on dist-test-slave-zkpd"
server_key: "4e83b0630f0009ff33674bf75b977b18"
server_key_iv: "41876159452554a7e2f030dede62c9a3"
server_key_version: "encryptionkey@0"
I20260811 11:56:19.491720 17214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:56:19.510988 17214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:56:19.511626 17214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:56:19.511816 17214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:56:19.512391 17214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:56:19.513286 17214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:56:19.513365 17214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:19.513430 17214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:56:19.513482 17214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:19.522068 17214 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:33207
I20260811 11:56:19.522107 17350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:33207 every 8 connection(s)
I20260811 11:56:19.522951 17214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:56:19.531551 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 17214
I20260811 11:56:19.531634 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance
I20260811 11:56:19.531949 6718 external_mini_cluster.cc:1428] Setting key 64a99a49252a23d5194d61dd71bd5132
I20260811 11:56:19.532373 16906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:19.524811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:43999 (local address 127.6.143.190:36315)
0811 11:56:19.524968 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:19.524972 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:19.526048 (+ 1076us) server_negotiation.cc:407] Connection header received
0811 11:56:19.527614 (+ 1566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:19.527617 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:19.527677 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:19.527785 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:19.528701 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:19.529232 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:19.530358 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:19.530588 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:19.530701 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:19.531832 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:56:19.531838 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:56:19.531845 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:56:19.531858 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:56:19.531876 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:56:19.531885 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:56:19.531996 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 11:56:19.532167 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 11:56:19.532230 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 11:56:19.533372 17354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:19.525223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:36315 (local address 127.6.143.131:43999)
0811 11:56:19.525892 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:19.525934 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:19.527390 (+ 1456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:19.527986 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:19.527993 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:19.528079 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:19.528554 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:19.528567 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:19.529366 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:19.529374 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:19.530226 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:19.530236 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:19.530743 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:19.530747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:19.530911 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:19.531528 (+ 617us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:56:19.531530 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:19.531569 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:56:19.531671 (+ 102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:56:19.531998 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:56:19.532010 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:56:19.532018 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 11:56:19.532165 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 11:56:19.532442 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":115,"threads_started":1}
I20260811 11:56:19.534468 17351 heartbeater.cc:347] Connected to a master server at 127.6.143.190:36315
I20260811 11:56:19.534701 17351 heartbeater.cc:464] Registering TS with master...
I20260811 11:56:19.535199 17351 heartbeater.cc:511] Master 127.6.143.190:36315 requested a full tablet report, sending...
I20260811 11:56:19.536229 16860 ts_manager.cc:194] Registered new tserver with Master: 9081b85b5df84f5cb812f5f1a5d859b1 (127.6.143.131:33207)
I20260811 11:56:19.536790 16860 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.143.131:43999
I20260811 11:56:19.545650 6718 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 11:56:19.548410 16906 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:19.547399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44166 (local address 127.6.143.190:36315)
0811 11:56:19.547538 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:19.547543 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:19.547716 (+ 173us) server_negotiation.cc:407] Connection header received
0811 11:56:19.547809 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:19.547811 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:19.547848 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:19.547982 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:19.548261 (+ 279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44166: BlockingRecv error: recv got EOF from 127.0.0.1:44166 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 11:56:19.548507 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 16928
I20260811 11:56:19.559011 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 17068
I20260811 11:56:19.566676 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 17214
I20260811 11:56:19.573925 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 16828
2026-08-11T11:56:19Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (900 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T11:56:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:56:19Z Disabled control of system clock
I20260811 11:56:19.605852 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:39407
--webserver_interface=127.6.143.190
--webserver_port=0
--builtin_ntp_servers=127.6.143.148:39561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:39407
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 11:56:19.721239 17364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:56:19.721505 17364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:56:19.721542 17364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:56:19.725559 17364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:56:19.725622 17364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:56:19.725637 17364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:56:19.725687 17364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:56:19.725718 17364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:56:19.730693 17364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:39561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:39407
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:39407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:56:19.731914 17364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:56:19.733410 17364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:56:19.742108 17370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:56:19.742213 17364 server_base.cc:1034] running on GCE node
W20260811 11:56:19.742096 17369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:19.742118 17372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:56:19.743033 17364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:56:19.744211 17364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:56:19.745406 17364 hybrid_clock.cc:648] HybridClock initialized: now 1786449379745385 us; error 32 us; skew 500 ppm
I20260811 11:56:19.747225 17364 webserver.cc:460] Webserver started at http://127.6.143.190:46597/ using document root <none> and password file <none>
I20260811 11:56:19.747826 17364 fs_manager.cc:359] Metadata directory not provided
I20260811 11:56:19.747964 17364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:56:19.748205 17364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:56:19.750243 17364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "3cd86422dfea461e956c62ff7194c2de"
format_stamp: "Formatted at 2026-08-11 11:56:19 on dist-test-slave-zkpd"
server_key: "1f439770c95c3263f9108b6e3d237f57"
server_key_iv: "545a72c23a9f055ac85a07e3df0bffc9"
server_key_version: "encryptionkey@0"
I20260811 11:56:19.750814 17364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "3cd86422dfea461e956c62ff7194c2de"
format_stamp: "Formatted at 2026-08-11 11:56:19 on dist-test-slave-zkpd"
server_key: "1f439770c95c3263f9108b6e3d237f57"
server_key_iv: "545a72c23a9f055ac85a07e3df0bffc9"
server_key_version: "encryptionkey@0"
I20260811 11:56:19.754436 17364 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 11:56:19.756834 17378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:19.757977 17364 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:56:19.758121 17364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "3cd86422dfea461e956c62ff7194c2de"
format_stamp: "Formatted at 2026-08-11 11:56:19 on dist-test-slave-zkpd"
server_key: "1f439770c95c3263f9108b6e3d237f57"
server_key_iv: "545a72c23a9f055ac85a07e3df0bffc9"
server_key_version: "encryptionkey@0"
I20260811 11:56:19.758232 17364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:56:19.762719 17364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:56:19.763325 17364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:56:19.763509 17364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:56:19.770471 17364 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:39407
I20260811 11:56:19.770481 17434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:39407 every 8 connection(s)
I20260811 11:56:19.771353 17364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:56:19.771694 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 17364
I20260811 11:56:19.771842 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:56:19.772137 6718 external_mini_cluster.cc:1428] Setting key 3569bd5ae3761849d33aa1441709557d
I20260811 11:56:19.774727 17435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:56:19.778330 17438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:19.773568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33130 (local address 127.6.143.190:39407)
0811 11:56:19.774792 (+ 1224us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:19.774808 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:19.774874 (+ 66us) server_negotiation.cc:407] Connection header received
0811 11:56:19.775541 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:19.775559 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:19.775942 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:19.776296 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:19.777112 (+ 816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:56:19.777116 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:56:19.777127 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:56:19.777138 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:56:19.777193 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:56:19.777208 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:56:19.777361 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 11:56:19.777444 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 11:56:19.777678 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":886,"thread_start_us":720,"threads_started":1}
I20260811 11:56:19.781080 17435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cd86422dfea461e956c62ff7194c2de: Bootstrap starting.
I20260811 11:56:19.783250 17435 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cd86422dfea461e956c62ff7194c2de: Neither blocks nor log segments found. Creating new log.
I20260811 11:56:19.784075 17435 log.cc:824] T 00000000000000000000000000000000 P 3cd86422dfea461e956c62ff7194c2de: Log is configured to *not* fsync() on all Append() calls
I20260811 11:56:19.786087 17435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cd86422dfea461e956c62ff7194c2de: No bootstrap required, opened a new log
I20260811 11:56:19.789747 17435 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cd86422dfea461e956c62ff7194c2de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cd86422dfea461e956c62ff7194c2de" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39407 } }
I20260811 11:56:19.790002 17435 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cd86422dfea461e956c62ff7194c2de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:56:19.790084 17435 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cd86422dfea461e956c62ff7194c2de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cd86422dfea461e956c62ff7194c2de, State: Initialized, Role: FOLLOWER
I20260811 11:56:19.790519 17435 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cd86422dfea461e956c62ff7194c2de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cd86422dfea461e956c62ff7194c2de" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39407 } }
I20260811 11:56:19.790678 17435 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3cd86422dfea461e956c62ff7194c2de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:56:19.790771 17435 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cd86422dfea461e956c62ff7194c2de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:56:19.790875 17435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cd86422dfea461e956c62ff7194c2de [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:56:19.791756 17435 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cd86422dfea461e956c62ff7194c2de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cd86422dfea461e956c62ff7194c2de" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39407 } }
I20260811 11:56:19.792135 17435 leader_election.cc:302] T 00000000000000000000000000000000 P 3cd86422dfea461e956c62ff7194c2de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3cd86422dfea461e956c62ff7194c2de; no voters:
I20260811 11:56:19.792470 17435 leader_election.cc:288] T 00000000000000000000000000000000 P 3cd86422dfea461e956c62ff7194c2de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:56:19.792593 17441 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cd86422dfea461e956c62ff7194c2de [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:56:19.792888 17441 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cd86422dfea461e956c62ff7194c2de [term 1 LEADER]: Becoming Leader. State: Replica: 3cd86422dfea461e956c62ff7194c2de, State: Running, Role: LEADER
I20260811 11:56:19.793272 17441 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cd86422dfea461e956c62ff7194c2de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cd86422dfea461e956c62ff7194c2de" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39407 } }
I20260811 11:56:19.793797 17435 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cd86422dfea461e956c62ff7194c2de [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:56:19.795068 17443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cd86422dfea461e956c62ff7194c2de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cd86422dfea461e956c62ff7194c2de. Latest consensus state: current_term: 1 leader_uuid: "3cd86422dfea461e956c62ff7194c2de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cd86422dfea461e956c62ff7194c2de" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39407 } } }
I20260811 11:56:19.795120 17442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cd86422dfea461e956c62ff7194c2de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3cd86422dfea461e956c62ff7194c2de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cd86422dfea461e956c62ff7194c2de" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39407 } } }
I20260811 11:56:19.795189 17443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cd86422dfea461e956c62ff7194c2de [sys.catalog]: This master's current role is: LEADER
I20260811 11:56:19.795220 17442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cd86422dfea461e956c62ff7194c2de [sys.catalog]: This master's current role is: LEADER
I20260811 11:56:19.795594 17450 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:56:19.799049 17450 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:56:19.804900 17450 catalog_manager.cc:1347] Generated new cluster ID: b25bf5caee994f039950236b58825996
I20260811 11:56:19.804977 17450 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:56:19.816128 17450 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:56:19.816429 17450 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:56:19.819830 17450 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3cd86422dfea461e956c62ff7194c2de: Generated new TSK 0
I20260811 11:56:19.820530 17450 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:56:19.830536 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:0
--local_ip_for_outbound_sockets=127.6.143.129
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:39407
--builtin_ntp_servers=127.6.143.148:39561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 11:56:19.945233 17461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:56:19.945465 17461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:56:19.945539 17461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:56:19.949805 17461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:56:19.949862 17461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:56:19.949918 17461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:56:19.954631 17461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:39561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:39407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:56:19.955705 17461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:56:19.957250 17461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:56:19.968781 17467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:19.969002 17466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:19.968786 17469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:56:19.969329 17461 server_base.cc:1034] running on GCE node
I20260811 11:56:19.969832 17461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:56:19.970608 17461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:56:19.971774 17461 hybrid_clock.cc:648] HybridClock initialized: now 1786449379971759 us; error 27 us; skew 500 ppm
I20260811 11:56:19.973521 17461 webserver.cc:460] Webserver started at http://127.6.143.129:38773/ using document root <none> and password file <none>
I20260811 11:56:19.974085 17461 fs_manager.cc:359] Metadata directory not provided
I20260811 11:56:19.974146 17461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:56:19.974313 17461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:56:19.976397 17461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/data/instance:
uuid: "9335e0d774ab42ac9c700458cbf874b9"
format_stamp: "Formatted at 2026-08-11 11:56:19 on dist-test-slave-zkpd"
server_key: "cb56ac023cc0749e00db71fde50f9f9d"
server_key_iv: "932c3a1f84d2ecbed26e57d710a5b4c7"
server_key_version: "encryptionkey@0"
I20260811 11:56:19.976954 17461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance:
uuid: "9335e0d774ab42ac9c700458cbf874b9"
format_stamp: "Formatted at 2026-08-11 11:56:19 on dist-test-slave-zkpd"
server_key: "cb56ac023cc0749e00db71fde50f9f9d"
server_key_iv: "932c3a1f84d2ecbed26e57d710a5b4c7"
server_key_version: "encryptionkey@0"
I20260811 11:56:19.980706 17461 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 11:56:19.983192 17475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:19.984440 17461 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:56:19.984539 17461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "9335e0d774ab42ac9c700458cbf874b9"
format_stamp: "Formatted at 2026-08-11 11:56:19 on dist-test-slave-zkpd"
server_key: "cb56ac023cc0749e00db71fde50f9f9d"
server_key_iv: "932c3a1f84d2ecbed26e57d710a5b4c7"
server_key_version: "encryptionkey@0"
I20260811 11:56:19.984619 17461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:56:19.989028 17461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:56:19.989562 17461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:56:19.989699 17461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:56:19.990170 17461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:56:19.991009 17461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:56:19.991055 17461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:19.991092 17461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:56:19.991114 17461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:19.999768 17461 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:42503
I20260811 11:56:19.999836 17597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:42503 every 8 connection(s)
I20260811 11:56:20.000777 17461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:56:20.006824 17438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:20.002657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:42765 (local address 127.6.143.190:39407)
0811 11:56:20.002831 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:20.002836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:20.003946 (+ 1110us) server_negotiation.cc:407] Connection header received
0811 11:56:20.005287 (+ 1341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:20.005289 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:20.005326 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:20.005402 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:20.006439 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:56:20.006442 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:56:20.006444 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:56:20.006455 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:56:20.006474 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:56:20.006481 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:56:20.006557 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 11:56:20.006679 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 11:56:20.006715 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 11:56:20.007572 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 17461
I20260811 11:56:20.007660 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance
I20260811 11:56:20.007776 17600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:20.003096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39407 (local address 127.6.143.129:42765)
0811 11:56:20.003774 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:20.003815 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:20.005095 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:20.005579 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:20.005586 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:20.005676 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:20.006193 (+ 517us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:56:20.006194 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:20.006234 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:56:20.006301 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:56:20.006556 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:56:20.006563 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:56:20.006568 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:56:20.006696 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 11:56:20.006912 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":117,"threads_started":1}
I20260811 11:56:20.008057 6718 external_mini_cluster.cc:1428] Setting key e17c862816ea5eb42af15bd7cf25b5b7
I20260811 11:56:20.009228 17598 heartbeater.cc:347] Connected to a master server at 127.6.143.190:39407
I20260811 11:56:20.009501 17598 heartbeater.cc:464] Registering TS with master...
I20260811 11:56:20.009989 17598 heartbeater.cc:511] Master 127.6.143.190:39407 requested a full tablet report, sending...
I20260811 11:56:20.010162 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:0
--local_ip_for_outbound_sockets=127.6.143.130
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:39407
--builtin_ntp_servers=127.6.143.148:39561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 11:56:20.011698 17396 ts_manager.cc:194] Registered new tserver with Master: 9335e0d774ab42ac9c700458cbf874b9 (127.6.143.129:42503)
W20260811 11:56:20.126533 17601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:56:20.126792 17601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:56:20.126830 17601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:56:20.131058 17601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:56:20.131112 17601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:56:20.131209 17601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:56:20.136041 17601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:39561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:39407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:56:20.137076 17601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:56:20.138468 17601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:56:20.147976 17607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:20.147971 17606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:56:20.148135 17601 server_base.cc:1034] running on GCE node
W20260811 11:56:20.148300 17609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:56:20.148727 17601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:56:20.149330 17601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:56:20.150494 17601 hybrid_clock.cc:648] HybridClock initialized: now 1786449380150467 us; error 47 us; skew 500 ppm
I20260811 11:56:20.152338 17601 webserver.cc:460] Webserver started at http://127.6.143.130:35711/ using document root <none> and password file <none>
I20260811 11:56:20.152935 17601 fs_manager.cc:359] Metadata directory not provided
I20260811 11:56:20.152997 17601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:56:20.153252 17601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:56:20.155248 17601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/data/instance:
uuid: "766e381ea53c47d4bbf9cddaa827d3df"
format_stamp: "Formatted at 2026-08-11 11:56:20 on dist-test-slave-zkpd"
server_key: "60dae9bcf990cca10aa95a9c50102d19"
server_key_iv: "6ab1b84ba3653943ab68bb7eec4006ec"
server_key_version: "encryptionkey@0"
I20260811 11:56:20.155810 17601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance:
uuid: "766e381ea53c47d4bbf9cddaa827d3df"
format_stamp: "Formatted at 2026-08-11 11:56:20 on dist-test-slave-zkpd"
server_key: "60dae9bcf990cca10aa95a9c50102d19"
server_key_iv: "6ab1b84ba3653943ab68bb7eec4006ec"
server_key_version: "encryptionkey@0"
I20260811 11:56:20.159488 17601 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 11:56:20.162123 17615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:20.163348 17601 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:56:20.163487 17601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "766e381ea53c47d4bbf9cddaa827d3df"
format_stamp: "Formatted at 2026-08-11 11:56:20 on dist-test-slave-zkpd"
server_key: "60dae9bcf990cca10aa95a9c50102d19"
server_key_iv: "6ab1b84ba3653943ab68bb7eec4006ec"
server_key_version: "encryptionkey@0"
I20260811 11:56:20.163610 17601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:56:20.168103 17601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:56:20.168725 17601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:56:20.168918 17601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:56:20.169507 17601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:56:20.170401 17601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:56:20.170472 17601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:20.170562 17601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:56:20.170585 17601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:20.179302 17601 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:36585
I20260811 11:56:20.179322 17740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:36585 every 8 connection(s)
I20260811 11:56:20.180305 17601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
I20260811 11:56:20.186364 17438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:20.182228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:46941 (local address 127.6.143.190:39407)
0811 11:56:20.182364 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:20.182368 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:20.183445 (+ 1077us) server_negotiation.cc:407] Connection header received
0811 11:56:20.184797 (+ 1352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:20.184800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:20.184839 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:20.184921 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:20.185967 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:56:20.185970 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:56:20.185972 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:56:20.185984 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:56:20.186006 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:56:20.186015 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:56:20.186093 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 11:56:20.186215 (+ 122us) server_negotiation.cc:299] Negotiation successful
0811 11:56:20.186255 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 11:56:20.187211 17743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:20.182642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39407 (local address 127.6.143.130:46941)
0811 11:56:20.183290 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:20.183330 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:20.184596 (+ 1266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:20.185092 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:20.185098 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:20.185186 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:20.185708 (+ 522us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:56:20.185710 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:20.185750 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:56:20.185820 (+ 70us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:56:20.186089 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:56:20.186095 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:56:20.186100 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 11:56:20.186234 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 11:56:20.186429 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":114,"threads_started":1}
I20260811 11:56:20.188423 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 17601
I20260811 11:56:20.188454 17741 heartbeater.cc:347] Connected to a master server at 127.6.143.190:39407
I20260811 11:56:20.188556 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance
I20260811 11:56:20.188695 17741 heartbeater.cc:464] Registering TS with master...
I20260811 11:56:20.188879 6718 external_mini_cluster.cc:1428] Setting key 4af0c396d3bae68b208370b67a3a0733
I20260811 11:56:20.189175 17741 heartbeater.cc:511] Master 127.6.143.190:39407 requested a full tablet report, sending...
I20260811 11:56:20.190341 17396 ts_manager.cc:194] Registered new tserver with Master: 766e381ea53c47d4bbf9cddaa827d3df (127.6.143.130:36585)
I20260811 11:56:20.191023 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:0
--local_ip_for_outbound_sockets=127.6.143.131
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:39407
--builtin_ntp_servers=127.6.143.148:39561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 11:56:20.314613 17744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:56:20.314939 17744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:56:20.315032 17744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:56:20.319530 17744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:56:20.319643 17744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:56:20.319746 17744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:56:20.325114 17744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:39561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:39407
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:56:20.326299 17744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:56:20.327855 17744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:56:20.337857 17749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:20.337857 17750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:20.337899 17752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:56:20.338282 17744 server_base.cc:1034] running on GCE node
I20260811 11:56:20.338706 17744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:56:20.339385 17744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:56:20.340656 17744 hybrid_clock.cc:648] HybridClock initialized: now 1786449380340625 us; error 48 us; skew 500 ppm
I20260811 11:56:20.342577 17744 webserver.cc:460] Webserver started at http://127.6.143.131:35875/ using document root <none> and password file <none>
I20260811 11:56:20.343163 17744 fs_manager.cc:359] Metadata directory not provided
I20260811 11:56:20.343256 17744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:56:20.343494 17744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:56:20.345597 17744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/data/instance:
uuid: "a5bd0ba58ee140faa72961fcf7a57f6b"
format_stamp: "Formatted at 2026-08-11 11:56:20 on dist-test-slave-zkpd"
server_key: "ca0141056cca3319ca923a1307203259"
server_key_iv: "247f80efafad0c397f51ee997500dbc2"
server_key_version: "encryptionkey@0"
I20260811 11:56:20.346170 17744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance:
uuid: "a5bd0ba58ee140faa72961fcf7a57f6b"
format_stamp: "Formatted at 2026-08-11 11:56:20 on dist-test-slave-zkpd"
server_key: "ca0141056cca3319ca923a1307203259"
server_key_iv: "247f80efafad0c397f51ee997500dbc2"
server_key_version: "encryptionkey@0"
I20260811 11:56:20.349872 17744 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260811 11:56:20.352811 17760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:20.354055 17744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 11:56:20.354192 17744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "a5bd0ba58ee140faa72961fcf7a57f6b"
format_stamp: "Formatted at 2026-08-11 11:56:20 on dist-test-slave-zkpd"
server_key: "ca0141056cca3319ca923a1307203259"
server_key_iv: "247f80efafad0c397f51ee997500dbc2"
server_key_version: "encryptionkey@0"
I20260811 11:56:20.354311 17744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:56:20.358915 17744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:56:20.359505 17744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:56:20.359688 17744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:56:20.360289 17744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:56:20.361209 17744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:56:20.361280 17744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:20.361349 17744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:56:20.361399 17744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:20.370527 17744 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:40721
I20260811 11:56:20.370549 17887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:40721 every 8 connection(s)
I20260811 11:56:20.371397 17744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:56:20.377274 17438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:20.373203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:39129 (local address 127.6.143.190:39407)
0811 11:56:20.373356 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:20.373362 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:20.374453 (+ 1091us) server_negotiation.cc:407] Connection header received
0811 11:56:20.375695 (+ 1242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:20.375697 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:20.375735 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:20.375809 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:20.376914 (+ 1105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:56:20.376916 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:56:20.376919 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:56:20.376928 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:56:20.376946 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:56:20.376953 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:56:20.377025 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 11:56:20.377132 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 11:56:20.377169 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 11:56:20.377990 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 17744
I20260811 11:56:20.378082 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance
I20260811 11:56:20.378192 17890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:20.373649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39407 (local address 127.6.143.131:39129)
0811 11:56:20.374301 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:20.374341 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:20.375509 (+ 1168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:20.375969 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:20.375975 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:20.376060 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:20.376668 (+ 608us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:56:20.376669 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:20.376710 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:56:20.376779 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:56:20.377019 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:56:20.377024 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:56:20.377028 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:56:20.377150 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 11:56:20.377363 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":109,"threads_started":1}
I20260811 11:56:20.378417 6718 external_mini_cluster.cc:1428] Setting key e02b6b2f46e01933e0b810392d0a1873
I20260811 11:56:20.379485 17888 heartbeater.cc:347] Connected to a master server at 127.6.143.190:39407
I20260811 11:56:20.379765 17888 heartbeater.cc:464] Registering TS with master...
I20260811 11:56:20.380203 17888 heartbeater.cc:511] Master 127.6.143.190:39407 requested a full tablet report, sending...
I20260811 11:56:20.381300 17396 ts_manager.cc:194] Registered new tserver with Master: a5bd0ba58ee140faa72961fcf7a57f6b (127.6.143.131:40721)
I20260811 11:56:20.392340 6718 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 11:56:20.395417 17438 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:56:20.394336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38040 (local address 127.6.143.190:39407)
0811 11:56:20.394483 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:20.394489 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:20.394697 (+ 208us) server_negotiation.cc:407] Connection header received
0811 11:56:20.394909 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:20.394913 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:20.394956 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:20.395043 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:20.395298 (+ 255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38040: BlockingRecv error: recv got EOF from 127.0.0.1:38040 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 11:56:20.395632 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 17461
I20260811 11:56:20.406088 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 17601
I20260811 11:56:20.416471 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 17744
I20260811 11:56:20.424906 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 17364
2026-08-11T11:56:20Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (851 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:56:20 dist-test-slave-zkpd krb5kdc[17898](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:56:20 dist-test-slave-zkpd krb5kdc[17898](info): set up 2 sockets
Aug 11 11:56:20 dist-test-slave-zkpd krb5kdc[17898](info): commencing operation
krb5kdc: starting...
W20260811 11:56:21.509110 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:56:21 dist-test-slave-zkpd krb5kdc[17898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:56:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:56:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
I20260811 11:56:21.632546 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:33137
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:34425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:33137
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:56:21.745699 17913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:56:21.745952 17913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:56:21.745993 17913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:56:21.750089 17913 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:56:21.750142 17913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:56:21.750161 17913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:56:21.750178 17913 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:56:21.750193 17913 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:56:21.755424 17913 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:34425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:33137
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:33137
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:56:21.756506 17913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:56:21.757516 17913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:56:21.766304 17920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:56:21.766435 17913 server_base.cc:1034] running on GCE node
W20260811 11:56:21.766335 17922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:21.766341 17919 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:56:21.767115 17913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:56:21.768322 17913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:56:21.769490 17913 hybrid_clock.cc:648] HybridClock initialized: now 1786449381769457 us; error 45 us; skew 500 ppm
Aug 11 11:56:21 dist-test-slave-zkpd krb5kdc[17898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449381, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:56:21.772265 17913 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:56:21.773165 17913 webserver.cc:460] Webserver started at http://127.6.143.190:39999/ using document root <none> and password file <none>
I20260811 11:56:21.773736 17913 fs_manager.cc:359] Metadata directory not provided
I20260811 11:56:21.773797 17913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:56:21.774027 17913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:56:21.776129 17913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "8f9797f122304f63847c24fe740559f0"
format_stamp: "Formatted at 2026-08-11 11:56:21 on dist-test-slave-zkpd"
server_key: "1ac6c2e096ae26d8c64b23c8a88b52c6"
server_key_iv: "9eb1511d3dd1141143666f5cfad8be8f"
server_key_version: "encryptionkey@0"
I20260811 11:56:21.776700 17913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "8f9797f122304f63847c24fe740559f0"
format_stamp: "Formatted at 2026-08-11 11:56:21 on dist-test-slave-zkpd"
server_key: "1ac6c2e096ae26d8c64b23c8a88b52c6"
server_key_iv: "9eb1511d3dd1141143666f5cfad8be8f"
server_key_version: "encryptionkey@0"
I20260811 11:56:21.780311 17913 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 11:56:21.782634 17929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:21.784011 17913 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:56:21.784137 17913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "8f9797f122304f63847c24fe740559f0"
format_stamp: "Formatted at 2026-08-11 11:56:21 on dist-test-slave-zkpd"
server_key: "1ac6c2e096ae26d8c64b23c8a88b52c6"
server_key_iv: "9eb1511d3dd1141143666f5cfad8be8f"
server_key_version: "encryptionkey@0"
I20260811 11:56:21.784250 17913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:56:21.801474 17913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:56:21.805013 17913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:56:21.805233 17913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:56:21.812844 17913 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:33137
I20260811 11:56:21.813156 17986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:33137 every 8 connection(s)
I20260811 11:56:21.814018 17913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:56:21.816992 17988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:56:21.819077 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 17913
I20260811 11:56:21.819185 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:56:21.819483 6718 external_mini_cluster.cc:1428] Setting key 30ece8cabc840cf2ec6109e282a178ec
I20260811 11:56:21.823204 17988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f9797f122304f63847c24fe740559f0: Bootstrap starting.
I20260811 11:56:21.825589 17988 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f9797f122304f63847c24fe740559f0: Neither blocks nor log segments found. Creating new log.
I20260811 11:56:21.826377 17988 log.cc:824] T 00000000000000000000000000000000 P 8f9797f122304f63847c24fe740559f0: Log is configured to *not* fsync() on all Append() calls
Aug 11 11:56:21 dist-test-slave-zkpd krb5kdc[17898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449381, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:56:21.828687 17988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f9797f122304f63847c24fe740559f0: No bootstrap required, opened a new log
I20260811 11:56:21.831981 17988 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f9797f122304f63847c24fe740559f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f9797f122304f63847c24fe740559f0" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 33137 } }
I20260811 11:56:21.832206 17988 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f9797f122304f63847c24fe740559f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:56:21.832298 17988 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f9797f122304f63847c24fe740559f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f9797f122304f63847c24fe740559f0, State: Initialized, Role: FOLLOWER
I20260811 11:56:21.832768 17988 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f9797f122304f63847c24fe740559f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f9797f122304f63847c24fe740559f0" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 33137 } }
I20260811 11:56:21.832926 17988 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f9797f122304f63847c24fe740559f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:56:21.833024 17988 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f9797f122304f63847c24fe740559f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:56:21.833134 17988 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f9797f122304f63847c24fe740559f0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:56:21.834152 17988 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f9797f122304f63847c24fe740559f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f9797f122304f63847c24fe740559f0" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 33137 } }
I20260811 11:56:21.834501 17988 leader_election.cc:302] T 00000000000000000000000000000000 P 8f9797f122304f63847c24fe740559f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f9797f122304f63847c24fe740559f0; no voters:
I20260811 11:56:21.834543 17992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:21.821014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58396 (local address 127.6.143.190:33137)
0811 11:56:21.821498 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:21.821514 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:21.821551 (+ 37us) server_negotiation.cc:407] Connection header received
0811 11:56:21.822297 (+ 746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:21.822317 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:21.822668 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:21.823015 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:21.823821 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:21.824875 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:21.825713 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:21.826198 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:21.826375 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:21.829224 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:56:21.829229 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:56:21.829244 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:56:21.829271 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:56:21.831395 (+ 2124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:56:21.831882 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:56:21.831888 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:56:21.831939 (+ 51us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:56:21.832068 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:56:21.832378 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:56:21.832381 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:56:21.832383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:56:21.832860 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:56:21.833094 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0811 11:56:21.833396 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 11:56:21.833740 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":142,"threads_started":1}
I20260811 11:56:21.834818 17988 leader_election.cc:288] T 00000000000000000000000000000000 P 8f9797f122304f63847c24fe740559f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:56:21.835176 17995 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f9797f122304f63847c24fe740559f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:56:21.835354 17995 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f9797f122304f63847c24fe740559f0 [term 1 LEADER]: Becoming Leader. State: Replica: 8f9797f122304f63847c24fe740559f0, State: Running, Role: LEADER
I20260811 11:56:21.835737 17995 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f9797f122304f63847c24fe740559f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f9797f122304f63847c24fe740559f0" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 33137 } }
I20260811 11:56:21.836184 17988 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f9797f122304f63847c24fe740559f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:56:21.836966 17995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f9797f122304f63847c24fe740559f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f9797f122304f63847c24fe740559f0. Latest consensus state: current_term: 1 leader_uuid: "8f9797f122304f63847c24fe740559f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f9797f122304f63847c24fe740559f0" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 33137 } } }
I20260811 11:56:21.837188 17995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f9797f122304f63847c24fe740559f0 [sys.catalog]: This master's current role is: LEADER
I20260811 11:56:21.837415 17996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f9797f122304f63847c24fe740559f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f9797f122304f63847c24fe740559f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f9797f122304f63847c24fe740559f0" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 33137 } } }
I20260811 11:56:21.837518 17996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f9797f122304f63847c24fe740559f0 [sys.catalog]: This master's current role is: LEADER
I20260811 11:56:21.837881 18003 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:56:21.840713 18003 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:56:21.847322 18003 catalog_manager.cc:1347] Generated new cluster ID: 589565e3579a4001baae9af9e14024a0
I20260811 11:56:21.847411 18003 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:56:21.869596 18003 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:56:21.870497 18003 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:56:21.876782 18003 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f9797f122304f63847c24fe740559f0: Generated new TSK 0
I20260811 11:56:21.877472 18003 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
I20260811 11:56:21.908854 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:0
--local_ip_for_outbound_sockets=127.6.143.129
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:33137
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:34425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:56:22.048336 18019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:56:22.048592 18019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:56:22.048632 18019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:56:22.052423 18019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:56:22.052469 18019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:56:22.052521 18019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:56:22.057542 18019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:34425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:33137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:56:22.058615 18019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:56:22.059633 18019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:56:22.069375 18024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:22.069387 18025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:22.069386 18027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:56:22.069581 18019 server_base.cc:1034] running on GCE node
I20260811 11:56:22.069993 18019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:56:22.070627 18019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:56:22.071820 18019 hybrid_clock.cc:648] HybridClock initialized: now 1786449382071804 us; error 27 us; skew 500 ppm
Aug 11 11:56:22 dist-test-slave-zkpd krb5kdc[17898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449382, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:56:22.074515 18019 init.cc:377] Logged in from keytab as kudu/127.6.143.129@KRBTEST.COM (short username kudu)
I20260811 11:56:22.075295 18019 webserver.cc:460] Webserver started at http://127.6.143.129:39689/ using document root <none> and password file <none>
I20260811 11:56:22.075932 18019 fs_manager.cc:359] Metadata directory not provided
I20260811 11:56:22.076007 18019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:56:22.076184 18019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:56:22.078109 18019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/data/instance:
uuid: "db62989122de4c2194592114fdc14553"
format_stamp: "Formatted at 2026-08-11 11:56:22 on dist-test-slave-zkpd"
server_key: "7c3d3b3a4d40868df82531f0054cc94d"
server_key_iv: "46977471b0ebdb9d51888a3426a33de5"
server_key_version: "encryptionkey@0"
I20260811 11:56:22.078608 18019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance:
uuid: "db62989122de4c2194592114fdc14553"
format_stamp: "Formatted at 2026-08-11 11:56:22 on dist-test-slave-zkpd"
server_key: "7c3d3b3a4d40868df82531f0054cc94d"
server_key_iv: "46977471b0ebdb9d51888a3426a33de5"
server_key_version: "encryptionkey@0"
I20260811 11:56:22.082269 18019 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 11:56:22.084671 18034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:22.085912 18019 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 11:56:22.086014 18019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "db62989122de4c2194592114fdc14553"
format_stamp: "Formatted at 2026-08-11 11:56:22 on dist-test-slave-zkpd"
server_key: "7c3d3b3a4d40868df82531f0054cc94d"
server_key_iv: "46977471b0ebdb9d51888a3426a33de5"
server_key_version: "encryptionkey@0"
I20260811 11:56:22.086092 18019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:56:22.102641 18019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:56:22.105321 18019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:56:22.105474 18019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:56:22.105968 18019 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:56:22.106781 18019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:56:22.106827 18019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:22.106865 18019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:56:22.106880 18019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:22.115332 18019 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:36565
I20260811 11:56:22.115363 18155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:36565 every 8 connection(s)
I20260811 11:56:22.116287 18019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
Aug 11 11:56:22 dist-test-slave-zkpd krb5kdc[17898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449382, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:56:22.126585 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 18019
I20260811 11:56:22.126713 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance
I20260811 11:56:22.127085 6718 external_mini_cluster.cc:1428] Setting key 56171110676aaca7d20f1bda2f66e367
I20260811 11:56:22.130255 17992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:22.118043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:55495 (local address 127.6.143.190:33137)
0811 11:56:22.118172 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:22.118176 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:22.119067 (+ 891us) server_negotiation.cc:407] Connection header received
0811 11:56:22.120627 (+ 1560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:22.120630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:22.120678 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:22.120751 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:22.121844 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.122355 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.123441 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.123668 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.123742 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:22.126111 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:56:22.126114 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:56:22.126116 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:56:22.126139 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:56:22.127717 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:56:22.128279 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:56:22.128283 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:56:22.128285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:56:22.128373 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:56:22.128776 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:56:22.128779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:56:22.128780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:56:22.129018 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:56:22.129121 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 11:56:22.129903 (+ 782us) server_negotiation.cc:299] Negotiation successful
0811 11:56:22.130125 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 11:56:22.131080 18158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:22.118401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:33137 (local address 127.6.143.129:55495)
0811 11:56:22.118903 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:22.118958 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:22.120435 (+ 1477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:22.120919 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:22.120925 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:22.121258 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:22.121717 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:22.121731 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.122473 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.122476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:22.123322 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:22.123330 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.123792 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.123795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:22.124022 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:22.124536 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:22.124552 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:22.125951 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:56:22.127841 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:56:22.127848 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:56:22.127861 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:56:22.128168 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:56:22.128477 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:56:22.128481 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:56:22.128483 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:56:22.128659 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:56:22.129163 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:56:22.129174 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:56:22.129508 (+ 334us) client_negotiation.cc:769] Sending connection context
0811 11:56:22.129745 (+ 237us) client_negotiation.cc:240] Negotiation successful
0811 11:56:22.130101 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":104,"threads_started":1}
I20260811 11:56:22.132388 18156 heartbeater.cc:347] Connected to a master server at 127.6.143.190:33137
I20260811 11:56:22.132639 18156 heartbeater.cc:464] Registering TS with master...
I20260811 11:56:22.133107 18156 heartbeater.cc:511] Master 127.6.143.190:33137 requested a full tablet report, sending...
I20260811 11:56:22.134650 17947 ts_manager.cc:194] Registered new tserver with Master: db62989122de4c2194592114fdc14553 (127.6.143.129:36565)
I20260811 11:56:22.135808 17947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.129@KRBTEST.COM'} at 127.6.143.129:55495
WARNING: no policy specified for kudu/127.6.143.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
I20260811 11:56:22.154584 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:0
--local_ip_for_outbound_sockets=127.6.143.130
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:33137
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:34425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:56:22.267279 18161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:56:22.267524 18161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:56:22.267583 18161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:56:22.271582 18161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:56:22.271639 18161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:56:22.271700 18161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:56:22.276865 18161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:34425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:33137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:56:22.277786 18161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:56:22.278738 18161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:56:22.288412 18169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:22.288415 18167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:22.288414 18166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:56:22.288681 18161 server_base.cc:1034] running on GCE node
I20260811 11:56:22.289206 18161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:56:22.289883 18161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:56:22.291074 18161 hybrid_clock.cc:648] HybridClock initialized: now 1786449382291050 us; error 31 us; skew 500 ppm
Aug 11 11:56:22 dist-test-slave-zkpd krb5kdc[17898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449382, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:56:22.293833 18161 init.cc:377] Logged in from keytab as kudu/127.6.143.130@KRBTEST.COM (short username kudu)
I20260811 11:56:22.294713 18161 webserver.cc:460] Webserver started at http://127.6.143.130:34575/ using document root <none> and password file <none>
I20260811 11:56:22.295310 18161 fs_manager.cc:359] Metadata directory not provided
I20260811 11:56:22.295403 18161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:56:22.295650 18161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:56:22.297744 18161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/data/instance:
uuid: "d9c7623bc84e4b87ab91f93f27b769f3"
format_stamp: "Formatted at 2026-08-11 11:56:22 on dist-test-slave-zkpd"
server_key: "22d2cee48e3301e0bb0f6607274de9c9"
server_key_iv: "3edd40422d273a5a7db9353d3511cc2a"
server_key_version: "encryptionkey@0"
I20260811 11:56:22.298326 18161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance:
uuid: "d9c7623bc84e4b87ab91f93f27b769f3"
format_stamp: "Formatted at 2026-08-11 11:56:22 on dist-test-slave-zkpd"
server_key: "22d2cee48e3301e0bb0f6607274de9c9"
server_key_iv: "3edd40422d273a5a7db9353d3511cc2a"
server_key_version: "encryptionkey@0"
I20260811 11:56:22.302050 18161 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 11:56:22.304559 18176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:22.305783 18161 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:56:22.305939 18161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "d9c7623bc84e4b87ab91f93f27b769f3"
format_stamp: "Formatted at 2026-08-11 11:56:22 on dist-test-slave-zkpd"
server_key: "22d2cee48e3301e0bb0f6607274de9c9"
server_key_iv: "3edd40422d273a5a7db9353d3511cc2a"
server_key_version: "encryptionkey@0"
I20260811 11:56:22.306053 18161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:56:22.336642 18161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:56:22.339973 18161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:56:22.340180 18161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:56:22.340824 18161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:56:22.341691 18161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:56:22.341763 18161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:22.341830 18161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:56:22.341883 18161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:22.350420 18161 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:45487
I20260811 11:56:22.350448 18295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:45487 every 8 connection(s)
I20260811 11:56:22.351339 18161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
Aug 11 11:56:22 dist-test-slave-zkpd krb5kdc[17898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449382, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:56:22.361204 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 18161
I20260811 11:56:22.361326 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance
I20260811 11:56:22.361626 6718 external_mini_cluster.cc:1428] Setting key 08f8e4cea4192bca91254c2d0d67c3e3
I20260811 11:56:22.364881 17992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:22.353126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:39669 (local address 127.6.143.190:33137)
0811 11:56:22.353267 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:22.353272 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:22.354179 (+ 907us) server_negotiation.cc:407] Connection header received
0811 11:56:22.355477 (+ 1298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:22.355480 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:22.355528 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:22.355601 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:22.356786 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.357305 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.358398 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.358605 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.358682 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:22.361009 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:56:22.361012 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:56:22.361014 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:56:22.361050 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:56:22.362601 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:56:22.363178 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:56:22.363181 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:56:22.363183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:56:22.363279 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:56:22.363607 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:56:22.363613 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:56:22.363618 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:56:22.363873 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:56:22.364003 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 11:56:22.364483 (+ 480us) server_negotiation.cc:299] Negotiation successful
0811 11:56:22.364630 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 11:56:22.365500 18300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:22.353545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:33137 (local address 127.6.143.130:39669)
0811 11:56:22.354029 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:22.354071 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:22.355280 (+ 1209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:22.355735 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:22.355742 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:22.356177 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:22.356655 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:22.356668 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.357431 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.357433 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:22.358256 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:22.358265 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.358734 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.358736 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:22.358881 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:22.359375 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:22.359390 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:22.360841 (+ 1451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:56:22.362711 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:56:22.362715 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:56:22.362726 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:56:22.363049 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:56:22.363379 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:56:22.363381 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:56:22.363382 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:56:22.363508 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:56:22.363990 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:56:22.363995 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:56:22.364248 (+ 253us) client_negotiation.cc:769] Sending connection context
0811 11:56:22.364403 (+ 155us) client_negotiation.cc:240] Negotiation successful
0811 11:56:22.364641 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":138,"threads_started":1}
I20260811 11:56:22.366670 18296 heartbeater.cc:347] Connected to a master server at 127.6.143.190:33137
I20260811 11:56:22.366905 18296 heartbeater.cc:464] Registering TS with master...
I20260811 11:56:22.367331 18296 heartbeater.cc:511] Master 127.6.143.190:33137 requested a full tablet report, sending...
I20260811 11:56:22.368384 17947 ts_manager.cc:194] Registered new tserver with Master: d9c7623bc84e4b87ab91f93f27b769f3 (127.6.143.130:45487)
I20260811 11:56:22.368943 17947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.130@KRBTEST.COM'} at 127.6.143.130:39669
WARNING: no policy specified for kudu/127.6.143.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
I20260811 11:56:22.391436 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:0
--local_ip_for_outbound_sockets=127.6.143.131
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:33137
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:34425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:56:22.503248 18303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:56:22.503506 18303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:56:22.503567 18303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:56:22.507488 18303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:56:22.507545 18303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:56:22.507642 18303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:56:22.512850 18303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:34425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:33137
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:56:22.513872 18303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:56:22.514863 18303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:56:22.524241 18308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:56:22.524308 18309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:56:22.524384 18303 server_base.cc:1034] running on GCE node
W20260811 11:56:22.524259 18311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:56:22.524886 18303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:56:22.525481 18303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:56:22.526669 18303 hybrid_clock.cc:648] HybridClock initialized: now 1786449382526659 us; error 31 us; skew 500 ppm
Aug 11 11:56:22 dist-test-slave-zkpd krb5kdc[17898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449382, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:56:22.529400 18303 init.cc:377] Logged in from keytab as kudu/127.6.143.131@KRBTEST.COM (short username kudu)
I20260811 11:56:22.530246 18303 webserver.cc:460] Webserver started at http://127.6.143.131:45343/ using document root <none> and password file <none>
I20260811 11:56:22.530794 18303 fs_manager.cc:359] Metadata directory not provided
I20260811 11:56:22.530884 18303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:56:22.531116 18303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:56:22.533115 18303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/data/instance:
uuid: "3d848043df92496fbfea24d1ee1131bc"
format_stamp: "Formatted at 2026-08-11 11:56:22 on dist-test-slave-zkpd"
server_key: "dda0774d382156fd84fe495000b9b224"
server_key_iv: "4b69334d2be5b2d618255ded59610935"
server_key_version: "encryptionkey@0"
I20260811 11:56:22.533689 18303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance:
uuid: "3d848043df92496fbfea24d1ee1131bc"
format_stamp: "Formatted at 2026-08-11 11:56:22 on dist-test-slave-zkpd"
server_key: "dda0774d382156fd84fe495000b9b224"
server_key_iv: "4b69334d2be5b2d618255ded59610935"
server_key_version: "encryptionkey@0"
I20260811 11:56:22.537356 18303 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 11:56:22.539664 18319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:22.540865 18303 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:56:22.541004 18303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "3d848043df92496fbfea24d1ee1131bc"
format_stamp: "Formatted at 2026-08-11 11:56:22 on dist-test-slave-zkpd"
server_key: "dda0774d382156fd84fe495000b9b224"
server_key_iv: "4b69334d2be5b2d618255ded59610935"
server_key_version: "encryptionkey@0"
I20260811 11:56:22.541119 18303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:56:22.550560 18303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:56:22.553272 18303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:56:22.553473 18303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:56:22.554049 18303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:56:22.554840 18303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:56:22.554932 18303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:22.555006 18303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:56:22.555042 18303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:56:22.563948 18303 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:36371
I20260811 11:56:22.563961 18444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:36371 every 8 connection(s)
I20260811 11:56:22.564844 18303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:56:22.568006 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 18303
I20260811 11:56:22.568131 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance
I20260811 11:56:22.568426 6718 external_mini_cluster.cc:1428] Setting key f78a5d67120b7cd7aed4637a2a93980e
Aug 11 11:56:22 dist-test-slave-zkpd krb5kdc[17898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449382, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:56:22.582454 17992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:22.566576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:41643 (local address 127.6.143.190:33137)
0811 11:56:22.566709 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:22.566713 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:22.567444 (+ 731us) server_negotiation.cc:407] Connection header received
0811 11:56:22.569134 (+ 1690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:22.569137 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:22.569201 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:22.569304 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:22.570458 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.570972 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.572195 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.572420 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.572532 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:22.579111 (+ 6579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:56:22.579115 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:56:22.579117 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:56:22.579140 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:56:22.580391 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:56:22.580928 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:56:22.580932 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:56:22.580933 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:56:22.581008 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:56:22.581352 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:56:22.581369 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:56:22.581370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:56:22.581553 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:56:22.581660 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 11:56:22.582145 (+ 485us) server_negotiation.cc:299] Negotiation successful
0811 11:56:22.582263 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 11:56:22.583176 18448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:22.566865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:33137 (local address 127.6.143.131:41643)
0811 11:56:22.567296 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:22.567336 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:22.568896 (+ 1560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:22.569488 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:22.569495 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:22.569835 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:56:22.570290 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:22.570302 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.571099 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.571103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:22.572069 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:22.572079 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.572581 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.572585 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:22.572755 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:22.577424 (+ 4669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:56:22.577440 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:56:22.578942 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:56:22.580511 (+ 1569us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:56:22.580517 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:56:22.580533 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:56:22.580788 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:56:22.581103 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:56:22.581106 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:56:22.581108 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:56:22.581246 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:56:22.581660 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:56:22.581668 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:56:22.581908 (+ 240us) client_negotiation.cc:769] Sending connection context
0811 11:56:22.582087 (+ 179us) client_negotiation.cc:240] Negotiation successful
0811 11:56:22.582393 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":109,"threads_started":1}
I20260811 11:56:22.584309 18445 heartbeater.cc:347] Connected to a master server at 127.6.143.190:33137
I20260811 11:56:22.584518 18445 heartbeater.cc:464] Registering TS with master...
I20260811 11:56:22.584937 18445 heartbeater.cc:511] Master 127.6.143.190:33137 requested a full tablet report, sending...
I20260811 11:56:22.585954 17947 ts_manager.cc:194] Registered new tserver with Master: 3d848043df92496fbfea24d1ee1131bc (127.6.143.131:36371)
I20260811 11:56:22.586509 17947 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.131@KRBTEST.COM'} at 127.6.143.131:41643
I20260811 11:56:22.593789 6718 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:56:22.602250 17992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:22.595677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58412 (local address 127.6.143.190:33137)
0811 11:56:22.595793 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:22.595797 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:22.595990 (+ 193us) server_negotiation.cc:407] Connection header received
0811 11:56:22.596089 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:22.596091 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:22.596126 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:22.596189 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:56:22.596823 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.597298 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.598085 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.598280 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.598353 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:22.599538 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:56:22.599541 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:56:22.599543 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:56:22.599564 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:56:22.600831 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:56:22.601205 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:56:22.601208 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:56:22.601209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:56:22.601280 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:56:22.601504 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:56:22.601506 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:56:22.601507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:56:22.601696 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:56:22.601794 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 11:56:22.602001 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 11:56:22.602085 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 11:56:22.604784 17947 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58412:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:56:22.607597 17947 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:56:22.621740 18457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:22.615754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:36565 (local address 127.0.0.1:49826)
0811 11:56:22.616561 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:22.616607 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:22.616797 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:22.617581 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:22.617587 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:22.617623 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:56:22.617814 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:22.617824 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.618948 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.618954 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:22.620251 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:22.620262 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.621222 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.621225 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:22.621329 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:22.621364 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 11:56:22.621481 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 11:56:22.621602 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":644,"mutex_wait_us":33,"spinlock_wait_cycles":4736,"thread_start_us":79,"threads_started":1}
I20260811 11:56:22.621749 18458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:22.615926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49826 (local address 127.6.143.129:36565)
0811 11:56:22.616495 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:22.616514 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:22.616703 (+ 189us) server_negotiation.cc:407] Connection header received
0811 11:56:22.617002 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:22.617009 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:22.617354 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:22.617488 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:56:22.617969 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.618813 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.620379 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.621094 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.621220 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:22.621368 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 11:56:22.621512 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 11:56:22.621601 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":98,"threads_started":1}
I20260811 11:56:22.621968 18460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:22.615985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:45487 (local address 127.0.0.1:54532)
0811 11:56:22.616646 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:22.616664 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:22.616795 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:22.617587 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:22.617590 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:22.617625 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:56:22.617860 (+ 235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:22.617868 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.619364 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.619367 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:22.620458 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:22.620467 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.621563 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.621567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:22.621661 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:22.621684 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 11:56:22.621733 (+ 49us) client_negotiation.cc:240] Negotiation successful
0811 11:56:22.621800 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"mutex_wait_us":45,"spinlock_wait_cycles":7040,"thread_start_us":82,"threads_started":1}
I20260811 11:56:22.622519 18459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:22.616201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:36371 (local address 127.0.0.1:60514)
0811 11:56:22.617214 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:22.617228 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:22.617316 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:22.617989 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:22.617992 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:22.618013 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:56:22.618097 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:22.618103 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.619451 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.619456 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:22.620642 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:22.620653 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.622104 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.622111 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:22.622214 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:22.622244 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 11:56:22.622322 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 11:56:22.622397 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":957,"thread_start_us":110,"threads_started":1}
I20260811 11:56:22.622609 18462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:22.616604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60514 (local address 127.6.143.131:36371)
0811 11:56:22.617087 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:22.617098 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:22.617425 (+ 327us) server_negotiation.cc:407] Connection header received
0811 11:56:22.617502 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:22.617507 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:22.617719 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:22.617872 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:56:22.618222 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.619309 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.620770 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.621556 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.621648 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:22.621782 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 11:56:22.622377 (+ 595us) server_negotiation.cc:299] Negotiation successful
0811 11:56:22.622479 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":103,"threads_started":1}
I20260811 11:56:22.622717 18461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:22.616567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54532 (local address 127.6.143.130:45487)
0811 11:56:22.617092 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:22.617107 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:22.617129 (+ 22us) server_negotiation.cc:407] Connection header received
0811 11:56:22.617204 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:22.617209 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:22.617438 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:22.617699 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:56:22.618013 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.619202 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.620579 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.621414 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.622268 (+ 854us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:22.622421 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 11:56:22.622471 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 11:56:22.622572 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":116,"threads_started":1}
I20260811 11:56:22.624975 18086 tablet_service.cc:1467] Processing CreateTablet for tablet e54efc7f61b9428dbde4d1ba0ccadd74 (DEFAULT_TABLE table=test-table [id=45760454dbf340e090c218a308fd7a8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:56:22.625970 18086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e54efc7f61b9428dbde4d1ba0ccadd74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:56:22.625590 18372 tablet_service.cc:1467] Processing CreateTablet for tablet e54efc7f61b9428dbde4d1ba0ccadd74 (DEFAULT_TABLE table=test-table [id=45760454dbf340e090c218a308fd7a8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:56:22.625590 18225 tablet_service.cc:1467] Processing CreateTablet for tablet e54efc7f61b9428dbde4d1ba0ccadd74 (DEFAULT_TABLE table=test-table [id=45760454dbf340e090c218a308fd7a8e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:56:22.626609 18225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e54efc7f61b9428dbde4d1ba0ccadd74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:56:22.626618 18372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e54efc7f61b9428dbde4d1ba0ccadd74. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:56:22.632478 18463 tablet_bootstrap.cc:492] T e54efc7f61b9428dbde4d1ba0ccadd74 P db62989122de4c2194592114fdc14553: Bootstrap starting.
I20260811 11:56:22.634218 18464 tablet_bootstrap.cc:492] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc: Bootstrap starting.
I20260811 11:56:22.634218 18465 tablet_bootstrap.cc:492] T e54efc7f61b9428dbde4d1ba0ccadd74 P d9c7623bc84e4b87ab91f93f27b769f3: Bootstrap starting.
I20260811 11:56:22.634841 18463 tablet_bootstrap.cc:654] T e54efc7f61b9428dbde4d1ba0ccadd74 P db62989122de4c2194592114fdc14553: Neither blocks nor log segments found. Creating new log.
I20260811 11:56:22.635609 18463 log.cc:824] T e54efc7f61b9428dbde4d1ba0ccadd74 P db62989122de4c2194592114fdc14553: Log is configured to *not* fsync() on all Append() calls
I20260811 11:56:22.636483 18464 tablet_bootstrap.cc:654] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc: Neither blocks nor log segments found. Creating new log.
I20260811 11:56:22.637106 18465 tablet_bootstrap.cc:654] T e54efc7f61b9428dbde4d1ba0ccadd74 P d9c7623bc84e4b87ab91f93f27b769f3: Neither blocks nor log segments found. Creating new log.
I20260811 11:56:22.637712 18463 tablet_bootstrap.cc:492] T e54efc7f61b9428dbde4d1ba0ccadd74 P db62989122de4c2194592114fdc14553: No bootstrap required, opened a new log
I20260811 11:56:22.637921 18464 log.cc:824] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc: Log is configured to *not* fsync() on all Append() calls
I20260811 11:56:22.637967 18463 ts_tablet_manager.cc:1397] T e54efc7f61b9428dbde4d1ba0ccadd74 P db62989122de4c2194592114fdc14553: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260811 11:56:22.637996 18465 log.cc:824] T e54efc7f61b9428dbde4d1ba0ccadd74 P d9c7623bc84e4b87ab91f93f27b769f3: Log is configured to *not* fsync() on all Append() calls
I20260811 11:56:22.640038 18464 tablet_bootstrap.cc:492] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc: No bootstrap required, opened a new log
I20260811 11:56:22.640321 18464 ts_tablet_manager.cc:1397] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260811 11:56:22.640383 18465 tablet_bootstrap.cc:492] T e54efc7f61b9428dbde4d1ba0ccadd74 P d9c7623bc84e4b87ab91f93f27b769f3: No bootstrap required, opened a new log
I20260811 11:56:22.640555 18465 ts_tablet_manager.cc:1397] T e54efc7f61b9428dbde4d1ba0ccadd74 P d9c7623bc84e4b87ab91f93f27b769f3: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 11:56:22.643154 18465 raft_consensus.cc:348] T e54efc7f61b9428dbde4d1ba0ccadd74 P d9c7623bc84e4b87ab91f93f27b769f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db62989122de4c2194592114fdc14553" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 36565 } } peers { permanent_uuid: "d9c7623bc84e4b87ab91f93f27b769f3" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 45487 } } peers { permanent_uuid: "3d848043df92496fbfea24d1ee1131bc" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36371 } }
I20260811 11:56:22.643348 18465 raft_consensus.cc:374] T e54efc7f61b9428dbde4d1ba0ccadd74 P d9c7623bc84e4b87ab91f93f27b769f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:56:22.643393 18465 raft_consensus.cc:729] T e54efc7f61b9428dbde4d1ba0ccadd74 P d9c7623bc84e4b87ab91f93f27b769f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9c7623bc84e4b87ab91f93f27b769f3, State: Initialized, Role: FOLLOWER
I20260811 11:56:22.643299 18463 raft_consensus.cc:348] T e54efc7f61b9428dbde4d1ba0ccadd74 P db62989122de4c2194592114fdc14553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db62989122de4c2194592114fdc14553" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 36565 } } peers { permanent_uuid: "d9c7623bc84e4b87ab91f93f27b769f3" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 45487 } } peers { permanent_uuid: "3d848043df92496fbfea24d1ee1131bc" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36371 } }
I20260811 11:56:22.643556 18463 raft_consensus.cc:374] T e54efc7f61b9428dbde4d1ba0ccadd74 P db62989122de4c2194592114fdc14553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:56:22.643615 18463 raft_consensus.cc:729] T e54efc7f61b9428dbde4d1ba0ccadd74 P db62989122de4c2194592114fdc14553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db62989122de4c2194592114fdc14553, State: Initialized, Role: FOLLOWER
I20260811 11:56:22.643836 18465 consensus_queue.cc:260] T e54efc7f61b9428dbde4d1ba0ccadd74 P d9c7623bc84e4b87ab91f93f27b769f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db62989122de4c2194592114fdc14553" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 36565 } } peers { permanent_uuid: "d9c7623bc84e4b87ab91f93f27b769f3" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 45487 } } peers { permanent_uuid: "3d848043df92496fbfea24d1ee1131bc" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36371 } }
I20260811 11:56:22.644065 18463 consensus_queue.cc:260] T e54efc7f61b9428dbde4d1ba0ccadd74 P db62989122de4c2194592114fdc14553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db62989122de4c2194592114fdc14553" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 36565 } } peers { permanent_uuid: "d9c7623bc84e4b87ab91f93f27b769f3" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 45487 } } peers { permanent_uuid: "3d848043df92496fbfea24d1ee1131bc" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36371 } }
I20260811 11:56:22.644258 18464 raft_consensus.cc:348] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db62989122de4c2194592114fdc14553" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 36565 } } peers { permanent_uuid: "d9c7623bc84e4b87ab91f93f27b769f3" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 45487 } } peers { permanent_uuid: "3d848043df92496fbfea24d1ee1131bc" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36371 } }
I20260811 11:56:22.644558 18464 raft_consensus.cc:374] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:56:22.644635 18464 raft_consensus.cc:729] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d848043df92496fbfea24d1ee1131bc, State: Initialized, Role: FOLLOWER
I20260811 11:56:22.645051 18464 consensus_queue.cc:260] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db62989122de4c2194592114fdc14553" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 36565 } } peers { permanent_uuid: "d9c7623bc84e4b87ab91f93f27b769f3" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 45487 } } peers { permanent_uuid: "3d848043df92496fbfea24d1ee1131bc" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36371 } }
I20260811 11:56:22.645139 18156 heartbeater.cc:503] Master 127.6.143.190:33137 was elected leader, sending a full tablet report...
I20260811 11:56:22.645190 18463 ts_tablet_manager.cc:1428] T e54efc7f61b9428dbde4d1ba0ccadd74 P db62989122de4c2194592114fdc14553: Time spent starting tablet: real 0.007s user 0.005s sys 0.001s
I20260811 11:56:22.645339 18465 ts_tablet_manager.cc:1428] T e54efc7f61b9428dbde4d1ba0ccadd74 P d9c7623bc84e4b87ab91f93f27b769f3: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 11:56:22.645756 18296 heartbeater.cc:503] Master 127.6.143.190:33137 was elected leader, sending a full tablet report...
I20260811 11:56:22.645992 18464 ts_tablet_manager.cc:1428] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 11:56:22.646142 18445 heartbeater.cc:503] Master 127.6.143.190:33137 was elected leader, sending a full tablet report...
I20260811 11:56:22.649237 18472 raft_consensus.cc:482] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:56:22.649435 18472 raft_consensus.cc:504] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db62989122de4c2194592114fdc14553" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 36565 } } peers { permanent_uuid: "d9c7623bc84e4b87ab91f93f27b769f3" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 45487 } } peers { permanent_uuid: "3d848043df92496fbfea24d1ee1131bc" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36371 } }
I20260811 11:56:22.650682 18472 leader_election.cc:288] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db62989122de4c2194592114fdc14553 (127.6.143.129:36565), d9c7623bc84e4b87ab91f93f27b769f3 (127.6.143.130:45487)
I20260811 11:56:22.655381 18461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:22.651168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:50593 (local address 127.6.143.130:45487)
0811 11:56:22.651264 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:22.651267 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:22.651280 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:56:22.651327 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:22.651329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:22.651364 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:22.651461 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:56:22.651747 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.652380 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.654071 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.654679 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.654806 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:22.654858 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 11:56:22.655171 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 11:56:22.655246 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 11:56:22.655372 18458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:22.650956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:55581 (local address 127.6.143.129:36565)
0811 11:56:22.651167 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:22.651172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:22.651408 (+ 236us) server_negotiation.cc:407] Connection header received
0811 11:56:22.651522 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:22.651525 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:22.651572 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:22.651665 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:56:22.652016 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.652641 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.654071 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.654679 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.654806 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:22.654868 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 11:56:22.655171 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 11:56:22.655246 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260811 11:56:22.655422 18448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:22.650888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:45487 (local address 127.6.143.131:50593)
0811 11:56:22.651022 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:22.651041 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:22.651132 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:22.651472 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:22.651475 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:22.651495 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:56:22.651611 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:22.651615 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.652543 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.652548 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:22.653908 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:22.653920 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.654804 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.654808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:22.654990 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:22.655059 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 11:56:22.655135 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 11:56:22.655268 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":109}
I20260811 11:56:22.655436 18473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:22.650891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:36565 (local address 127.6.143.131:55581)
0811 11:56:22.651316 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:22.651327 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:56:22.651441 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:56:22.651684 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:56:22.651687 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:56:22.651720 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:56:22.651841 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:22.651846 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.652753 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.652756 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:22.653951 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:56:22.653962 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.654804 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.654808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:56:22.654986 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:22.655052 (+ 66us) client_negotiation.cc:769] Sending connection context
0811 11:56:22.655135 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 11:56:22.655265 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"mutex_wait_us":162,"thread_start_us":109,"threads_started":1}
I20260811 11:56:22.656113 18247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e54efc7f61b9428dbde4d1ba0ccadd74" candidate_uuid: "3d848043df92496fbfea24d1ee1131bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9c7623bc84e4b87ab91f93f27b769f3" is_pre_election: true
I20260811 11:56:22.656212 18108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e54efc7f61b9428dbde4d1ba0ccadd74" candidate_uuid: "3d848043df92496fbfea24d1ee1131bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db62989122de4c2194592114fdc14553" is_pre_election: true
I20260811 11:56:22.656477 18247 raft_consensus.cc:2454] T e54efc7f61b9428dbde4d1ba0ccadd74 P d9c7623bc84e4b87ab91f93f27b769f3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d848043df92496fbfea24d1ee1131bc in term 0.
I20260811 11:56:22.656497 18108 raft_consensus.cc:2454] T e54efc7f61b9428dbde4d1ba0ccadd74 P db62989122de4c2194592114fdc14553 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d848043df92496fbfea24d1ee1131bc in term 0.
I20260811 11:56:22.656978 18321 leader_election.cc:302] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 3d848043df92496fbfea24d1ee1131bc, d9c7623bc84e4b87ab91f93f27b769f3, db62989122de4c2194592114fdc14553; no voters:
I20260811 11:56:22.657240 18472 raft_consensus.cc:2781] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:56:22.657315 18472 raft_consensus.cc:482] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:56:22.657346 18472 raft_consensus.cc:3037] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:56:22.658412 18472 raft_consensus.cc:504] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db62989122de4c2194592114fdc14553" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 36565 } } peers { permanent_uuid: "d9c7623bc84e4b87ab91f93f27b769f3" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 45487 } } peers { permanent_uuid: "3d848043df92496fbfea24d1ee1131bc" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36371 } }
I20260811 11:56:22.658792 18472 leader_election.cc:288] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc [CANDIDATE]: Term 1 election: Requested vote from peers db62989122de4c2194592114fdc14553 (127.6.143.129:36565), d9c7623bc84e4b87ab91f93f27b769f3 (127.6.143.130:45487)
I20260811 11:56:22.659106 18108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e54efc7f61b9428dbde4d1ba0ccadd74" candidate_uuid: "3d848043df92496fbfea24d1ee1131bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db62989122de4c2194592114fdc14553"
I20260811 11:56:22.659142 18247 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e54efc7f61b9428dbde4d1ba0ccadd74" candidate_uuid: "3d848043df92496fbfea24d1ee1131bc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d9c7623bc84e4b87ab91f93f27b769f3"
I20260811 11:56:22.659246 18247 raft_consensus.cc:3037] T e54efc7f61b9428dbde4d1ba0ccadd74 P d9c7623bc84e4b87ab91f93f27b769f3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:56:22.659246 18108 raft_consensus.cc:3037] T e54efc7f61b9428dbde4d1ba0ccadd74 P db62989122de4c2194592114fdc14553 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:56:22.660579 18108 raft_consensus.cc:2454] T e54efc7f61b9428dbde4d1ba0ccadd74 P db62989122de4c2194592114fdc14553 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d848043df92496fbfea24d1ee1131bc in term 1.
I20260811 11:56:22.660579 18247 raft_consensus.cc:2454] T e54efc7f61b9428dbde4d1ba0ccadd74 P d9c7623bc84e4b87ab91f93f27b769f3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d848043df92496fbfea24d1ee1131bc in term 1.
I20260811 11:56:22.660881 18322 leader_election.cc:302] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d848043df92496fbfea24d1ee1131bc, d9c7623bc84e4b87ab91f93f27b769f3; no voters:
I20260811 11:56:22.661089 18472 raft_consensus.cc:2781] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:56:22.661294 18472 raft_consensus.cc:686] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc [term 1 LEADER]: Becoming Leader. State: Replica: 3d848043df92496fbfea24d1ee1131bc, State: Running, Role: LEADER
I20260811 11:56:22.661628 18472 consensus_queue.cc:237] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db62989122de4c2194592114fdc14553" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 36565 } } peers { permanent_uuid: "d9c7623bc84e4b87ab91f93f27b769f3" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 45487 } } peers { permanent_uuid: "3d848043df92496fbfea24d1ee1131bc" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36371 } }
I20260811 11:56:22.665078 17946 catalog_manager.cc:5495] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d848043df92496fbfea24d1ee1131bc (127.6.143.131). New cstate: current_term: 1 leader_uuid: "3d848043df92496fbfea24d1ee1131bc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db62989122de4c2194592114fdc14553" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 36565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9c7623bc84e4b87ab91f93f27b769f3" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 45487 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3d848043df92496fbfea24d1ee1131bc" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36371 } health_report { overall_health: HEALTHY } } }
I20260811 11:56:22.683941 18462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:22.680696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60518 (local address 127.6.143.131:36371)
0811 11:56:22.680828 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 11:56:22.680832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:56:22.680845 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:56:22.680958 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:56:22.680960 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:56:22.681015 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:56:22.681100 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:56:22.681315 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.681814 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.682607 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:56:22.682811 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:56:22.682883 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:56:22.683149 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:56:22.683566 (+ 417us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:56:22.683651 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 11:56:22.683712 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 11:56:22.683756 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 11:56:22.689900 18247 raft_consensus.cc:1261] T e54efc7f61b9428dbde4d1ba0ccadd74 P d9c7623bc84e4b87ab91f93f27b769f3 [term 1 FOLLOWER]: Refusing update from remote peer 3d848043df92496fbfea24d1ee1131bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:56:22.689916 18108 raft_consensus.cc:1261] T e54efc7f61b9428dbde4d1ba0ccadd74 P db62989122de4c2194592114fdc14553 [term 1 FOLLOWER]: Refusing update from remote peer 3d848043df92496fbfea24d1ee1131bc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:56:22.690665 18474 consensus_queue.cc:1035] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "db62989122de4c2194592114fdc14553" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 36565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:56:22.690807 18472 consensus_queue.cc:1035] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc [LEADER]: Connected to new peer: Peer: permanent_uuid: "d9c7623bc84e4b87ab91f93f27b769f3" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 45487 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:56:22.698638 18481 mvcc.cc:205] Tried to move back new op lower bound from 7317296671490510848 to 7317296671385001984. Current Snapshot: MvccSnapshot[applied={T|T < 7317296671490510848}]
I20260811 11:56:22.706636 17947 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:56:22.706811 17947 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58412:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:56:22.709263 17947 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8f9797f122304f63847c24fe740559f0: Sending DeleteTablet for 3 replicas of tablet e54efc7f61b9428dbde4d1ba0ccadd74
I20260811 11:56:22.710078 18086 tablet_service.cc:1514] Processing DeleteTablet for tablet e54efc7f61b9428dbde4d1ba0ccadd74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:56:22 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:49826
I20260811 11:56:22.710129 18225 tablet_service.cc:1514] Processing DeleteTablet for tablet e54efc7f61b9428dbde4d1ba0ccadd74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:56:22 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:54532
I20260811 11:56:22.710579 18372 tablet_service.cc:1514] Processing DeleteTablet for tablet e54efc7f61b9428dbde4d1ba0ccadd74 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:56:22 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:60514
I20260811 11:56:22.710582 18491 tablet_replica.cc:330] T e54efc7f61b9428dbde4d1ba0ccadd74 P db62989122de4c2194592114fdc14553: stopping tablet replica
I20260811 11:56:22.710930 18491 raft_consensus.cc:2229] T e54efc7f61b9428dbde4d1ba0ccadd74 P db62989122de4c2194592114fdc14553 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:56:22.711030 18493 tablet_replica.cc:330] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc: stopping tablet replica
I20260811 11:56:22.711167 18492 tablet_replica.cc:330] T e54efc7f61b9428dbde4d1ba0ccadd74 P d9c7623bc84e4b87ab91f93f27b769f3: stopping tablet replica
I20260811 11:56:22.711287 18491 raft_consensus.cc:2258] T e54efc7f61b9428dbde4d1ba0ccadd74 P db62989122de4c2194592114fdc14553 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:56:22.711350 18493 raft_consensus.cc:2229] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:56:22.711380 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 18019
I20260811 11:56:22.712540 18493 raft_consensus.cc:2258] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:56:22.713773 18493 ts_tablet_manager.cc:1905] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:56:22.714377 18492 raft_consensus.cc:2229] T e54efc7f61b9428dbde4d1ba0ccadd74 P d9c7623bc84e4b87ab91f93f27b769f3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:56:22.714675 18492 raft_consensus.cc:2258] T e54efc7f61b9428dbde4d1ba0ccadd74 P d9c7623bc84e4b87ab91f93f27b769f3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:56:22.716068 18493 ts_tablet_manager.cc:1918] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:56:22.716183 18493 log.cc:1196] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-2/wal/wals/e54efc7f61b9428dbde4d1ba0ccadd74
I20260811 11:56:22.716221 18492 ts_tablet_manager.cc:1905] T e54efc7f61b9428dbde4d1ba0ccadd74 P d9c7623bc84e4b87ab91f93f27b769f3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:56:22.716507 18493 ts_tablet_manager.cc:1939] T e54efc7f61b9428dbde4d1ba0ccadd74 P 3d848043df92496fbfea24d1ee1131bc: Deleting consensus metadata
I20260811 11:56:22.717530 17930 catalog_manager.cc:4841] TS 3d848043df92496fbfea24d1ee1131bc (127.6.143.131:36371): tablet e54efc7f61b9428dbde4d1ba0ccadd74 (table test-table [id=45760454dbf340e090c218a308fd7a8e]) successfully deleted
I20260811 11:56:22.719621 18492 ts_tablet_manager.cc:1918] T e54efc7f61b9428dbde4d1ba0ccadd74 P d9c7623bc84e4b87ab91f93f27b769f3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:56:22.719729 18492 log.cc:1196] T e54efc7f61b9428dbde4d1ba0ccadd74 P d9c7623bc84e4b87ab91f93f27b769f3: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786449319379472-6718-0/minicluster-data/ts-1/wal/wals/e54efc7f61b9428dbde4d1ba0ccadd74
I20260811 11:56:22.720130 18492 ts_tablet_manager.cc:1939] T e54efc7f61b9428dbde4d1ba0ccadd74 P d9c7623bc84e4b87ab91f93f27b769f3: Deleting consensus metadata
I20260811 11:56:22.721405 17932 catalog_manager.cc:4841] TS d9c7623bc84e4b87ab91f93f27b769f3 (127.6.143.130:45487): tablet e54efc7f61b9428dbde4d1ba0ccadd74 (table test-table [id=45760454dbf340e090c218a308fd7a8e]) successfully deleted
W20260811 11:56:22.722158 17931 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.143.129:36565 (error 108)
I20260811 11:56:22.723053 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 18161
I20260811 11:56:22.723239 18459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:56:22.722931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:36565 (local address 127.0.0.1:49828)
0811 11:56:22.723056 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 11:56:22.723138 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.129:36565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 11:56:22.730825 17931 catalog_manager.cc:4584] TS db62989122de4c2194592114fdc14553 (127.6.143.129:36565): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e54efc7f61b9428dbde4d1ba0ccadd74: Network error: Client connection negotiation failed: client connection to 127.6.143.129:36565: connect: Connection refused (error 111)
I20260811 11:56:22.739398 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 18303
I20260811 11:56:22.746924 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 17913
2026-08-11T11:56:22Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2322 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:56:22 dist-test-slave-zkpd krb5kdc[18498](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:56:22 dist-test-slave-zkpd krb5kdc[18498](info): set up 2 sockets
Aug 11 11:56:22 dist-test-slave-zkpd krb5kdc[18498](info): commencing operation
krb5kdc: starting...
W20260811 11:56:23.830368 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:56:23 dist-test-slave-zkpd krb5kdc[18498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:56:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:56:23Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.6.143.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.6.143.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.6.143.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/rangeradmin_127.6.143.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.6.143.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/rangeradmin_127.6.143.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.6.143.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.6.143.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.6.143.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/rangerlookup_127.6.143.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.6.143.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/rangerlookup_127.6.143.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.143.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.6.143.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.143.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/HTTP_127.6.143.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.143.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/HTTP_127.6.143.148@KRBTEST.COM.keytab.
I20260811 11:56:24.006465 6718 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskEMvzen/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/postgres -l logfile start
2026-08-11 11:56:26.351 UTC [18529] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 11:56:26.351 UTC [18529] LOG: listening on IPv4 address "127.6.143.148", port 37419
2026-08-11 11:56:26.354 UTC [18529] LOG: listening on Unix socket "/tmp/.s.PGSQL.37419"
2026-08-11 11:56:26.375 UTC [18532] LOG: database system was shut down at 2026-08-11 11:56:25 UTC
2026-08-11 11:56:26.379 UTC [18529] LOG: database system is ready to accept connections
I20260811 11:56:27.381907 6718 mini_postgres.cc:96] Postgres bound to 37419
2026-08-11 11:56:27.389 UTC [18540] FATAL: database "slave" does not exist
127.6.143.148:37419 - accepting connections
I20260811 11:56:27.390146 6718 mini_ranger.cc:161] Starting Ranger...
I20260811 11:56:27.405557 6718 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 11:56:27.832700 6718 mini_ranger.cc:87] Created ranger Postgres database
I20260811 11:56:27.832782 6718 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ranger-admin
2026-08-11 11:56:28,114 [I] DB FLAVOR :POSTGRES
2026-08-11 11:56:28,114 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 11:56:28,114 [I] Checking connection..
2026-08-11 11:56:28,114 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:37419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 11:56:28,373 [I] Checking connection passed.
2026-08-11 11:56:28,374 [I] --------- Verifying version history table ---------
2026-08-11 11:56:28,374 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:37419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 11:56:28,635 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 11:56:28,635 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:37419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 11:56:28,878 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 11:56:28,878 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 11:56:28,878 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:37419/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 11:56:29.121 UTC [18613] WARNING: there is no transaction in progress
2026-08-11 11:56:29,127 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 11:56:29,128 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:37419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 11:56:29,376 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 11:56:29,377 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 11:56:29,377 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:37419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 11:56:29,639 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:37419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 11:56:29,878 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:37419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-zkpd.c.gcp-upstream.internal','N') ;"
2026-08-11 11:56:30,125 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:37419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 11:56:30,388 [I] Table x_portal_user does not exist in database ranger
2026-08-11 11:56:30,388 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:37419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 11:56:30,631 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 11:56:30,632 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:37419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 11:56:30,874 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:37419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 11:56:31,117 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 11:56:31,117 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:37419/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 11:56:32.011 UTC [18734] WARNING: there is no transaction in progress
2026-08-11 11:56:32.027 UTC [18734] WARNING: there is no transaction in progress
2026-08-11 11:56:32.042 UTC [18734] WARNING: there is no transaction in progress
2026-08-11 11:56:32.057 UTC [18734] WARNING: there is no transaction in progress
2026-08-11 11:56:32.074 UTC [18734] WARNING: there is no transaction in progress
2026-08-11 11:56:32.093 UTC [18734] WARNING: there is no transaction in progress
2026-08-11 11:56:32.154 UTC [18734] WARNING: there is no transaction in progress
2026-08-11 11:56:32.160 UTC [18734] WARNING: there is no transaction in progress
2026-08-11 11:56:32.174 UTC [18734] WARNING: there is no transaction in progress
2026-08-11 11:56:32.184 UTC [18734] WARNING: there is no transaction in progress
2026-08-11 11:56:32.194 UTC [18734] WARNING: there is no transaction in progress
2026-08-11 11:56:32.208 UTC [18734] WARNING: there is no transaction in progress
2026-08-11 11:56:32.217 UTC [18734] WARNING: there is no transaction in progress
2026-08-11 11:56:32.223 UTC [18734] WARNING: there is no transaction in progress
2026-08-11 11:56:32.230 UTC [18734] WARNING: there is no transaction in progress
2026-08-11 11:56:32.236 UTC [18734] WARNING: there is no transaction in progress
2026-08-11 11:56:32,840 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 11:56:32,840 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:37419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 11:56:33,068 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 11:56:33,068 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:37419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 11:56:33,327 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 11:56:33,327 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:37419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 11:56:33,589 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 11:56:33,589 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:37419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 11:56:33,839 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:37419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 11:56:34,097 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:37419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-zkpd.c.gcp-upstream.internal';"
2026-08-11 11:56:34,320 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 11:56:34,320 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:37419/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 11:56:34,558 [I] DB_PATCHES have already been applied
I20260811 11:56:34.563884 6718 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ranger-admin:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskEMvzen/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 11:56:34.564010 6718 mini_ranger.cc:193] Using host: 127.6.143.148
I20260811 11:56:34.568301 6718 mini_ranger.cc:227] Ranger admin URL: http://127.6.143.148:41457
Aug 11, 2026 11:56:34 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 11:56:34 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 11:56:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 11:56:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 11:56:34 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 11:56:35 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 11:56:35 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 11:56:35.716897 6718 mini_ranger.cc:160] Time spent starting Ranger: real 8.327s user 0.006s sys 0.006s
Aug 11, 2026 11:56:55 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 11:56:55 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 11, 2026 11:56:55 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 11:56:55 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 11:56:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 11:56:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 11:56:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 11:56:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 11:56:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 11:56:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 11:56:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 11:56:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 11:56:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 11:56:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 11:56:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 11:56:55 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 11:56:55 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 11:56:56 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 11:56:57.380378 6718 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.6.143.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.6.143.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.6.143.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/rangerkms_127.6.143.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.6.143.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/rangerkms_127.6.143.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.143.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.6.143.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.143.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/HTTP_127.6.143.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.143.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/HTTP_127.6.143.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 11:56:57 dist-test-slave-zkpd krb5kdc[18498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449417, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 11:56:57.483675 6718 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 11:56:57.493839 6718 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 11:56:58.269124 6718 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 11:56:58.269209 6718 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ranger-kms
I20260811 11:56:58.269301 6718 mini_ranger_kms.cc:227] Using postgres at 127.6.143.148:37419
2026-08-11 11:56:58,462 [I] DB FLAVOR :POSTGRES
2026-08-11 11:56:58,462 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 11:56:58,462 [I] Checking connection
2026-08-11 11:56:58,462 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:37419/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 11:56:58,704 [I] connection success
2026-08-11 11:56:58,704 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 11:56:58,704 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 11:56:58,704 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:37419/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 11:56:58,948 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 11:56:58,949 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 11:56:58,949 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 11:56:58,949 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:37419/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 11:56:59,231 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 11:56:59.722255 6718 mini_ranger_kms.cc:310] Created kms service
I20260811 11:56:59.923425 6718 mini_ranger_kms.cc:326] Created kudu user
I20260811 11:56:59.969605 6718 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 11:56:59 dist-test-slave-zkpd krb5kdc[18498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449417, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.6.143.148@KRBTEST.COM
I20260811 11:57:00.398187 6718 mini_ranger_kms.cc:384] Added ranger policy
I20260811 11:57:00.398355 6718 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ranger-kms:/tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 11:57:00.398406 6718 mini_ranger_kms.cc:240] Using host: 127.6.143.148
I20260811 11:57:00.403345 6718 mini_ranger_kms.cc:276] Ranger KMS PID: 18934
I20260811 11:57:00.403481 6718 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.6.143.148:55075
Aug 11, 2026 11:57:01 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 11:57:01 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 11:57:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 11:57:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 11:57:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 11:57:01 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 11:57:02 dist-test-slave-zkpd krb5kdc[18498](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786449422, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.143.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:57:02.766994 6718 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.283s user 0.003s sys 0.174s
I20260811 11:57:02.767149 6718 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 11:57:02.767280 6718 mini_ranger_kms.cc:401] 127.6.143.148:55075/kms/v1/keys
Aug 11 11:57:02 dist-test-slave-zkpd krb5kdc[18498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449422, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.143.148@KRBTEST.COM for HTTP/127.6.143.148@KRBTEST.COM
Aug 11 11:57:03 dist-test-slave-zkpd krb5kdc[18498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449422, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.143.148@KRBTEST.COM for HTTP/127.6.143.148@KRBTEST.COM
Aug 11 11:57:05 dist-test-slave-zkpd krb5kdc[18498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
I20260811 11:57:05.519043 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:37809
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:34391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:37809
--ranger_config_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.143.148:55075/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:57:05.633500 18992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:57:05.633865 18992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:57:05.633939 18992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:57:05.637907 18992 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:57:05.637960 18992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:57:05.637979 18992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:57:05.637995 18992 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:57:05.638010 18992 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:57:05.643348 18992 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:34391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/master-0/wal
--ranger_kms_url=127.6.143.148:55075/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:37809
--ranger_config_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:37809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18992
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:57:05.644445 18992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:57:05.645489 18992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:57:05.654166 18998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:57:05.654326 19000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:57:05.654179 18997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:57:05.654986 18992 server_base.cc:1034] running on GCE node
I20260811 11:57:05.655570 18992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:57:05.656790 18992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:57:05.657982 18992 hybrid_clock.cc:648] HybridClock initialized: now 1786449425657961 us; error 39 us; skew 500 ppm
Aug 11 11:57:05 dist-test-slave-zkpd krb5kdc[18498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449425, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:57:05.665251 18992 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:57:05.666458 18992 webserver.cc:460] Webserver started at http://127.6.143.190:39157/ using document root <none> and password file <none>
I20260811 11:57:05.667174 18992 fs_manager.cc:359] Metadata directory not provided
I20260811 11:57:05.667270 18992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:57:05.667503 18992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 11:57:05 dist-test-slave-zkpd krb5kdc[18498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449425, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for HTTP/127.6.143.148@KRBTEST.COM
I20260811 11:57:05.721438 18992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "c4d31edfc65448ef82fd92d82046a11b"
format_stamp: "Formatted at 2026-08-11 11:57:05 on dist-test-slave-zkpd"
server_key: "1d02dd0c6c5256df166917d7d587d8a8"
server_key_iv: "730dc941d2714e2ed275629df84a1ddf"
server_key_version: "kuduclusterkey@0"
I20260811 11:57:05.722074 18992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "c4d31edfc65448ef82fd92d82046a11b"
format_stamp: "Formatted at 2026-08-11 11:57:05 on dist-test-slave-zkpd"
server_key: "1d02dd0c6c5256df166917d7d587d8a8"
server_key_iv: "730dc941d2714e2ed275629df84a1ddf"
server_key_version: "kuduclusterkey@0"
I20260811 11:57:05.726169 18992 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 11:57:05.739760 19007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:05.741261 18992 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 11:57:05.741394 18992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "c4d31edfc65448ef82fd92d82046a11b"
format_stamp: "Formatted at 2026-08-11 11:57:05 on dist-test-slave-zkpd"
server_key: "1d02dd0c6c5256df166917d7d587d8a8"
server_key_iv: "730dc941d2714e2ed275629df84a1ddf"
server_key_version: "kuduclusterkey@0"
I20260811 11:57:05.741539 18992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:57:05.755493 18992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:57:05.758286 18992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:57:05.758514 18992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:57:05.766746 18992 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:37809
I20260811 11:57:05.766775 19063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:37809 every 8 connection(s)
I20260811 11:57:05.767727 18992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:57:05.770655 19064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:57:05.776288 19064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4d31edfc65448ef82fd92d82046a11b: Bootstrap starting.
I20260811 11:57:05.776751 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 18992
I20260811 11:57:05.776880 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:57:05.777155 6718 external_mini_cluster.cc:1428] Setting key 3728f72646787cf53c433dfdffadf282
I20260811 11:57:05.779093 19064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c4d31edfc65448ef82fd92d82046a11b: Neither blocks nor log segments found. Creating new log.
I20260811 11:57:05.780058 19064 log.cc:824] T 00000000000000000000000000000000 P c4d31edfc65448ef82fd92d82046a11b: Log is configured to *not* fsync() on all Append() calls
I20260811 11:57:05.784241 19064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c4d31edfc65448ef82fd92d82046a11b: No bootstrap required, opened a new log
Aug 11 11:57:05 dist-test-slave-zkpd krb5kdc[18498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449425, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:57:05.788362 19064 raft_consensus.cc:348] T 00000000000000000000000000000000 P c4d31edfc65448ef82fd92d82046a11b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4d31edfc65448ef82fd92d82046a11b" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 37809 } }
I20260811 11:57:05.788637 19064 raft_consensus.cc:374] T 00000000000000000000000000000000 P c4d31edfc65448ef82fd92d82046a11b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:57:05.788708 19064 raft_consensus.cc:729] T 00000000000000000000000000000000 P c4d31edfc65448ef82fd92d82046a11b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4d31edfc65448ef82fd92d82046a11b, State: Initialized, Role: FOLLOWER
I20260811 11:57:05.789256 19064 consensus_queue.cc:260] T 00000000000000000000000000000000 P c4d31edfc65448ef82fd92d82046a11b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4d31edfc65448ef82fd92d82046a11b" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 37809 } }
I20260811 11:57:05.789420 19064 raft_consensus.cc:388] T 00000000000000000000000000000000 P c4d31edfc65448ef82fd92d82046a11b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:57:05.789505 19064 raft_consensus.cc:482] T 00000000000000000000000000000000 P c4d31edfc65448ef82fd92d82046a11b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:57:05.789635 19064 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c4d31edfc65448ef82fd92d82046a11b [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:57:05.790669 19064 raft_consensus.cc:504] T 00000000000000000000000000000000 P c4d31edfc65448ef82fd92d82046a11b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4d31edfc65448ef82fd92d82046a11b" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 37809 } }
I20260811 11:57:05.791049 19064 leader_election.cc:302] T 00000000000000000000000000000000 P c4d31edfc65448ef82fd92d82046a11b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c4d31edfc65448ef82fd92d82046a11b; no voters:
I20260811 11:57:05.791400 19064 leader_election.cc:288] T 00000000000000000000000000000000 P c4d31edfc65448ef82fd92d82046a11b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:57:05.791936 19071 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c4d31edfc65448ef82fd92d82046a11b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:57:05.792578 19071 raft_consensus.cc:686] T 00000000000000000000000000000000 P c4d31edfc65448ef82fd92d82046a11b [term 1 LEADER]: Becoming Leader. State: Replica: c4d31edfc65448ef82fd92d82046a11b, State: Running, Role: LEADER
I20260811 11:57:05.792887 19064 sys_catalog.cc:564] T 00000000000000000000000000000000 P c4d31edfc65448ef82fd92d82046a11b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:57:05.792932 19071 consensus_queue.cc:237] T 00000000000000000000000000000000 P c4d31edfc65448ef82fd92d82046a11b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4d31edfc65448ef82fd92d82046a11b" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 37809 } }
I20260811 11:57:05.795245 19069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:05.778509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33136 (local address 127.6.143.190:37809)
0811 11:57:05.779019 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:05.779040 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:05.779086 (+ 46us) server_negotiation.cc:407] Connection header received
0811 11:57:05.779992 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:05.780017 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:05.780459 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:05.781872 (+ 1413us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:57:05.782297 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:05.783770 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:05.785048 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:05.785759 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:05.785944 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:05.789050 (+ 3106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:57:05.789056 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:57:05.789074 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:57:05.789105 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:57:05.791404 (+ 2299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:05.791947 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:05.791954 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:05.791961 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:05.792115 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:05.792455 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:05.792458 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:05.792460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:05.792958 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:57:05.793887 (+ 929us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:05.794037 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 11:57:05.794338 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":165,"threads_started":1}
I20260811 11:57:05.796205 19072 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4d31edfc65448ef82fd92d82046a11b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c4d31edfc65448ef82fd92d82046a11b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4d31edfc65448ef82fd92d82046a11b" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 37809 } } }
I20260811 11:57:05.796231 19073 sys_catalog.cc:455] T 00000000000000000000000000000000 P c4d31edfc65448ef82fd92d82046a11b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c4d31edfc65448ef82fd92d82046a11b. Latest consensus state: current_term: 1 leader_uuid: "c4d31edfc65448ef82fd92d82046a11b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c4d31edfc65448ef82fd92d82046a11b" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 37809 } } }
I20260811 11:57:05.796423 19073 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4d31edfc65448ef82fd92d82046a11b [sys.catalog]: This master's current role is: LEADER
I20260811 11:57:05.796711 19072 sys_catalog.cc:458] T 00000000000000000000000000000000 P c4d31edfc65448ef82fd92d82046a11b [sys.catalog]: This master's current role is: LEADER
I20260811 11:57:05.797843 19075 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:57:05.798208 19064 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 11:57:05.800734 19075 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:57:05.814378 19075 catalog_manager.cc:1347] Generated new cluster ID: 629810860642450cab1992e13ef81393
I20260811 11:57:05.814467 19075 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:57:05.823807 19075 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:57:05.825295 19075 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:57:05.854257 19075 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c4d31edfc65448ef82fd92d82046a11b: Generated new TSK 0
I20260811 11:57:05.855285 19075 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
I20260811 11:57:06.590934 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.143.148:55075/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:0
--local_ip_for_outbound_sockets=127.6.143.129
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:37809
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:34391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:57:06.746692 19114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:57:06.747135 19114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:57:06.747269 19114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:57:06.753419 19114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:57:06.753492 19114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:57:06.753578 19114 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:57:06.761480 19114 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:34391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.6.143.148:55075/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:37809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19114
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:57:06.762892 19114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:57:06.764361 19114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:57:06.777572 19124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:57:06.777661 19126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:57:06.778005 19123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:57:06.778507 19114 server_base.cc:1034] running on GCE node
I20260811 11:57:06.778925 19114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:57:06.779611 19114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:57:06.782871 19114 hybrid_clock.cc:648] HybridClock initialized: now 1786449426782835 us; error 44 us; skew 500 ppm
Aug 11 11:57:06 dist-test-slave-zkpd krb5kdc[18498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449426, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:57:06.786617 19114 init.cc:377] Logged in from keytab as kudu/127.6.143.129@KRBTEST.COM (short username kudu)
I20260811 11:57:06.787762 19114 webserver.cc:460] Webserver started at http://127.6.143.129:43373/ using document root <none> and password file <none>
I20260811 11:57:06.788527 19114 fs_manager.cc:359] Metadata directory not provided
I20260811 11:57:06.788636 19114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:57:06.788911 19114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 11:57:06 dist-test-slave-zkpd krb5kdc[18498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449426, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for HTTP/127.6.143.148@KRBTEST.COM
I20260811 11:57:06.807679 19114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-0/data/instance:
uuid: "db0f8ceec7b84b73accc75fab100f66b"
format_stamp: "Formatted at 2026-08-11 11:57:06 on dist-test-slave-zkpd"
server_key: "13514366ca30197795ee05605fed407f"
server_key_iv: "5f69969757f1159b1a843ee6981a39a1"
server_key_version: "kuduclusterkey@0"
I20260811 11:57:06.808483 19114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance:
uuid: "db0f8ceec7b84b73accc75fab100f66b"
format_stamp: "Formatted at 2026-08-11 11:57:06 on dist-test-slave-zkpd"
server_key: "13514366ca30197795ee05605fed407f"
server_key_iv: "5f69969757f1159b1a843ee6981a39a1"
server_key_version: "kuduclusterkey@0"
I20260811 11:57:06.816166 19114 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260811 11:57:06.844295 19133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:06.846274 19114 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.004s sys 0.000s
I20260811 11:57:06.846437 19114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "db0f8ceec7b84b73accc75fab100f66b"
format_stamp: "Formatted at 2026-08-11 11:57:06 on dist-test-slave-zkpd"
server_key: "13514366ca30197795ee05605fed407f"
server_key_iv: "5f69969757f1159b1a843ee6981a39a1"
server_key_version: "kuduclusterkey@0"
I20260811 11:57:06.846575 19114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:57:06.876530 19114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:57:06.880457 19114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:57:06.880746 19114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:57:06.881551 19114 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:57:06.882511 19114 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:57:06.882586 19114 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:06.882680 19114 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:57:06.882741 19114 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:06.894052 19114 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:41355
I20260811 11:57:06.894958 19114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:57:06.895653 19256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:41355 every 8 connection(s)
I20260811 11:57:06.901299 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 19114
I20260811 11:57:06.901434 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance
I20260811 11:57:06.901742 6718 external_mini_cluster.cc:1428] Setting key 397b694ce01a335dbfc42f4a75c76a55
Aug 11 11:57:06 dist-test-slave-zkpd krb5kdc[18498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449426, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.143.130@KRBTEST.COM; defaulting to no policy
I20260811 11:57:06.915133 19259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:06.897022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:47985 (local address 127.6.143.190:37809)
0811 11:57:06.898545 (+ 1523us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:06.898550 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:06.898569 (+ 19us) server_negotiation.cc:407] Connection header received
0811 11:57:06.899186 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:06.899189 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:06.899250 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:06.899344 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:57:06.900499 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:06.901104 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:06.902761 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:06.903071 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:06.903165 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:06.906353 (+ 3188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:57:06.906359 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:57:06.906364 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:57:06.906398 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:57:06.912752 (+ 6354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:06.913458 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:06.913461 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:06.913462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:06.913538 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:06.913910 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:06.913914 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:06.913916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:06.914189 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:57:06.914309 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:06.914816 (+ 507us) server_negotiation.cc:299] Negotiation successful
0811 11:57:06.914956 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1437,"thread_start_us":83,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.130@KRBTEST.COM" created.
I20260811 11:57:06.916456 19260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:06.897464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:37809 (local address 127.6.143.129:47985)
0811 11:57:06.898008 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:06.898064 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 11:57:06.898962 (+ 898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:57:06.899522 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:57:06.899530 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:57:06.899802 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:57:06.900352 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:06.900367 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:06.901272 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:06.901278 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:06.902611 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:06.902622 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:06.903341 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:06.903345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:06.903536 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:06.904264 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:57:06.904290 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:57:06.906149 (+ 1859us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:57:06.912897 (+ 6748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:06.912907 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:06.912926 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:06.913328 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:06.913637 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:06.913643 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:06.913645 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:06.913809 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:06.914308 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:57:06.914316 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:57:06.914568 (+ 252us) client_negotiation.cc:769] Sending connection context
0811 11:57:06.914774 (+ 206us) client_negotiation.cc:240] Negotiation successful
0811 11:57:06.915107 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":140,"threads_started":1}
I20260811 11:57:06.918172 19257 heartbeater.cc:347] Connected to a master server at 127.6.143.190:37809
I20260811 11:57:06.918459 19257 heartbeater.cc:464] Registering TS with master...
I20260811 11:57:06.918953 19257 heartbeater.cc:511] Master 127.6.143.190:37809 requested a full tablet report, sending...
I20260811 11:57:06.920774 19024 ts_manager.cc:194] Registered new tserver with Master: db0f8ceec7b84b73accc75fab100f66b (127.6.143.129:41355)
I20260811 11:57:06.921947 19024 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.129@KRBTEST.COM'} at 127.6.143.129:47985
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
I20260811 11:57:06.936388 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.143.148:55075/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:0
--local_ip_for_outbound_sockets=127.6.143.130
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:37809
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:34391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 11:57:06 dist-test-slave-zkpd krb5kdc[18498](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786449426, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 11:57:07.072701 19263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:57:07.072980 19263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:57:07.073053 19263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:57:07.077843 19263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:57:07.077922 19263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:57:07.078006 19263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:57:07.085631 19263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:34391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.6.143.148:55075/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:37809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:57:07.086905 19263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:57:07.088276 19263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:57:07.100641 19272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:57:07.100740 19274 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:57:07.100641 19271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:57:07.102074 19263 server_base.cc:1034] running on GCE node
I20260811 11:57:07.102577 19263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:57:07.103343 19263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:57:07.104533 19263 hybrid_clock.cc:648] HybridClock initialized: now 1786449427104499 us; error 48 us; skew 500 ppm
Aug 11 11:57:07 dist-test-slave-zkpd krb5kdc[18498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449427, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:57:07.108145 19263 init.cc:377] Logged in from keytab as kudu/127.6.143.130@KRBTEST.COM (short username kudu)
I20260811 11:57:07.109251 19263 webserver.cc:460] Webserver started at http://127.6.143.130:37489/ using document root <none> and password file <none>
I20260811 11:57:07.109990 19263 fs_manager.cc:359] Metadata directory not provided
I20260811 11:57:07.110126 19263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:57:07.110414 19263 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 11:57:07 dist-test-slave-zkpd krb5kdc[18498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449427, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for HTTP/127.6.143.148@KRBTEST.COM
I20260811 11:57:07.126189 19263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-1/data/instance:
uuid: "3f3eaa68ead84ac78bea4928da898fa3"
format_stamp: "Formatted at 2026-08-11 11:57:07 on dist-test-slave-zkpd"
server_key: "74333e6b29ecccf9fb94836609701f1b"
server_key_iv: "3375fb03bdd156559931e3c4b804a31f"
server_key_version: "kuduclusterkey@0"
I20260811 11:57:07.127080 19263 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance:
uuid: "3f3eaa68ead84ac78bea4928da898fa3"
format_stamp: "Formatted at 2026-08-11 11:57:07 on dist-test-slave-zkpd"
server_key: "74333e6b29ecccf9fb94836609701f1b"
server_key_iv: "3375fb03bdd156559931e3c4b804a31f"
server_key_version: "kuduclusterkey@0"
I20260811 11:57:07.134899 19263 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.000s
I20260811 11:57:07.157474 19281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:07.159233 19263 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.003s
I20260811 11:57:07.159366 19263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "3f3eaa68ead84ac78bea4928da898fa3"
format_stamp: "Formatted at 2026-08-11 11:57:07 on dist-test-slave-zkpd"
server_key: "74333e6b29ecccf9fb94836609701f1b"
server_key_iv: "3375fb03bdd156559931e3c4b804a31f"
server_key_version: "kuduclusterkey@0"
I20260811 11:57:07.159490 19263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:57:07.178596 19263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:57:07.182488 19263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:57:07.182794 19263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:57:07.183558 19263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:57:07.184749 19263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:57:07.184864 19263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:07.184970 19263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:57:07.185041 19263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:07.196605 19263 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:37721
I20260811 11:57:07.197962 19263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
I20260811 11:57:07.208009 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 19263
I20260811 11:57:07.208151 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance
I20260811 11:57:07.208501 6718 external_mini_cluster.cc:1428] Setting key 5e19144103c6e6d3d1bea94c235a3531
I20260811 11:57:07.212288 19401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:37721 every 8 connection(s)
WARNING: no policy specified for kudu/127.6.143.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.131@KRBTEST.COM" created.
Aug 11 11:57:07 dist-test-slave-zkpd krb5kdc[18498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449427, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
I20260811 11:57:07.243824 19259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:07.215827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:60713 (local address 127.6.143.190:37809)
0811 11:57:07.216003 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:07.216010 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:07.220466 (+ 4456us) server_negotiation.cc:407] Connection header received
0811 11:57:07.221915 (+ 1449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:07.221922 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:07.221986 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:07.222048 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:57:07.223458 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.224266 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.226097 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.226492 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.226588 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:07.238388 (+ 11800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:57:07.238393 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:57:07.238397 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:57:07.238438 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:57:07.240629 (+ 2191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:07.241456 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:07.241461 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:07.241463 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:07.241567 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:07.242106 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:07.242110 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:07.242112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:07.242449 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:57:07.242570 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:07.243420 (+ 850us) server_negotiation.cc:299] Negotiation successful
0811 11:57:07.243594 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 11:57:07.244170 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.143.148:55075/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:0
--local_ip_for_outbound_sockets=127.6.143.131
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:37809
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:34391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:57:07.256443 19405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:07.214725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:37809 (local address 127.6.143.130:60713)
0811 11:57:07.220274 (+ 5549us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:07.220340 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 11:57:07.221621 (+ 1281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:57:07.222407 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:57:07.222415 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:57:07.222781 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:57:07.223299 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:07.223312 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.224452 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.224459 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:07.225941 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:07.225957 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.226823 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.226828 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:07.227099 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:07.235784 (+ 8685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:57:07.235815 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:57:07.238032 (+ 2217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:57:07.240815 (+ 2783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:07.240825 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:07.240854 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:07.241325 (+ 471us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:07.241691 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:07.241695 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:07.241699 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:07.241958 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:07.242595 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:57:07.242606 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:57:07.243039 (+ 433us) client_negotiation.cc:769] Sending connection context
0811 11:57:07.254690 (+ 11651us) client_negotiation.cc:240] Negotiation successful
0811 11:57:07.255139 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5263,"thread_start_us":176,"threads_started":1}
I20260811 11:57:07.258365 19402 heartbeater.cc:347] Connected to a master server at 127.6.143.190:37809
I20260811 11:57:07.258621 19402 heartbeater.cc:464] Registering TS with master...
I20260811 11:57:07.259241 19402 heartbeater.cc:511] Master 127.6.143.190:37809 requested a full tablet report, sending...
I20260811 11:57:07.260490 19024 ts_manager.cc:194] Registered new tserver with Master: 3f3eaa68ead84ac78bea4928da898fa3 (127.6.143.130:37721)
I20260811 11:57:07.261267 19024 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.130@KRBTEST.COM'} at 127.6.143.130:60713
W20260811 11:57:07.434666 19407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:57:07.435127 19407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:57:07.435254 19407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:57:07.441182 19407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:57:07.441335 19407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:57:07.441468 19407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:57:07.449241 19407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:34391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.6.143.148:55075/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:37809
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:57:07.450691 19407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:57:07.452082 19407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:57:07.463510 19413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:57:07.469218 19414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:57:07.469518 19416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:57:07.471237 19407 server_base.cc:1034] running on GCE node
I20260811 11:57:07.471769 19407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:57:07.472643 19407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
Aug 11 11:57:07 dist-test-slave-zkpd krb5kdc[18498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449426, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for HTTP/127.6.143.148@KRBTEST.COM
I20260811 11:57:07.488835 19407 hybrid_clock.cc:648] HybridClock initialized: now 1786449427488729 us; error 105 us; skew 500 ppm
Aug 11 11:57:07 dist-test-slave-zkpd krb5kdc[18498](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449427, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:57:07.492372 19407 init.cc:377] Logged in from keytab as kudu/127.6.143.131@KRBTEST.COM (short username kudu)
I20260811 11:57:07.493594 19407 webserver.cc:460] Webserver started at http://127.6.143.131:32825/ using document root <none> and password file <none>
I20260811 11:57:07.494396 19407 fs_manager.cc:359] Metadata directory not provided
I20260811 11:57:07.494495 19407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:57:07.494781 19407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 11:57:07 dist-test-slave-zkpd krb5kdc[18498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449427, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for HTTP/127.6.143.148@KRBTEST.COM
I20260811 11:57:07.510972 19407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-2/data/instance:
uuid: "4b0a55200ec74366862314af9ecb8bde"
format_stamp: "Formatted at 2026-08-11 11:57:07 on dist-test-slave-zkpd"
server_key: "765f97491c83371872f8fc0b0e48143c"
server_key_iv: "900d40987f9747fa9e999108613eff21"
server_key_version: "kuduclusterkey@0"
I20260811 11:57:07.511730 19407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance:
uuid: "4b0a55200ec74366862314af9ecb8bde"
format_stamp: "Formatted at 2026-08-11 11:57:07 on dist-test-slave-zkpd"
server_key: "765f97491c83371872f8fc0b0e48143c"
server_key_iv: "900d40987f9747fa9e999108613eff21"
server_key_version: "kuduclusterkey@0"
I20260811 11:57:07.525453 19407 fs_manager.cc:691] Time spent creating directory manager: real 0.013s user 0.003s sys 0.001s
I20260811 11:57:07.556376 19423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:07.558310 19407 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.002s sys 0.000s
I20260811 11:57:07.558483 19407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "4b0a55200ec74366862314af9ecb8bde"
format_stamp: "Formatted at 2026-08-11 11:57:07 on dist-test-slave-zkpd"
server_key: "765f97491c83371872f8fc0b0e48143c"
server_key_iv: "900d40987f9747fa9e999108613eff21"
server_key_version: "kuduclusterkey@0"
I20260811 11:57:07.558652 19407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:57:07.599054 19407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:57:07.602814 19407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:57:07.603072 19407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:57:07.603789 19407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:57:07.604957 19407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:57:07.605062 19407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:07.605151 19407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:57:07.605211 19407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:07.615440 19407 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:45685
I20260811 11:57:07.616948 19407 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:57:07.617717 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 19407
I20260811 11:57:07.617849 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance
I20260811 11:57:07.618181 6718 external_mini_cluster.cc:1428] Setting key 5c75bd6336a91d3258d2d62124623e16
I20260811 11:57:07.624986 19548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:45685 every 8 connection(s)
Aug 11 11:57:07 dist-test-slave-zkpd krb5kdc[18498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449427, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:57:07.642441 19259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:07.628428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:39531 (local address 127.6.143.190:37809)
0811 11:57:07.628552 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:07.628557 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:07.629226 (+ 669us) server_negotiation.cc:407] Connection header received
0811 11:57:07.630431 (+ 1205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:07.630435 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:07.630490 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:07.630543 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:57:07.632186 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.633023 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.634741 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.635096 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.635169 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:07.638284 (+ 3115us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:57:07.638288 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:57:07.638291 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:57:07.638318 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:57:07.640123 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:07.640733 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:07.640737 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:07.640738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:07.640823 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:07.641210 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:07.641225 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:07.641227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:07.641432 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:57:07.641512 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:07.642156 (+ 644us) server_negotiation.cc:299] Negotiation successful
0811 11:57:07.642285 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 11:57:07.644443 19552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:07.627317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:37809 (local address 127.6.143.131:39531)
0811 11:57:07.629069 (+ 1752us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:07.629123 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 11:57:07.630171 (+ 1048us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:57:07.630954 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:57:07.630963 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:57:07.631331 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:57:07.632016 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:07.632038 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.633182 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.633185 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:07.634592 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:07.634604 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.635401 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.635405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:07.635605 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:07.636301 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:57:07.636322 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:57:07.638081 (+ 1759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:57:07.640262 (+ 2181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:07.640268 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:07.640285 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:07.640617 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:07.640930 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:07.640933 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:07.640935 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:07.641103 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:07.641578 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:57:07.641584 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:57:07.641886 (+ 302us) client_negotiation.cc:769] Sending connection context
0811 11:57:07.642901 (+ 1015us) client_negotiation.cc:240] Negotiation successful
0811 11:57:07.643357 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1516,"thread_start_us":140,"threads_started":1}
I20260811 11:57:07.646001 19550 heartbeater.cc:347] Connected to a master server at 127.6.143.190:37809
I20260811 11:57:07.646247 19550 heartbeater.cc:464] Registering TS with master...
I20260811 11:57:07.646847 19550 heartbeater.cc:511] Master 127.6.143.190:37809 requested a full tablet report, sending...
I20260811 11:57:07.648357 19024 ts_manager.cc:194] Registered new tserver with Master: 4b0a55200ec74366862314af9ecb8bde (127.6.143.131:45685)
I20260811 11:57:07.649026 19024 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.131@KRBTEST.COM'} at 127.6.143.131:39531
I20260811 11:57:07.650269 6718 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:57:07.662477 19259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:07.652777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33144 (local address 127.6.143.190:37809)
0811 11:57:07.652886 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:07.652890 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:07.653424 (+ 534us) server_negotiation.cc:407] Connection header received
0811 11:57:07.653591 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:07.653594 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:07.653633 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:07.653696 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:57:07.654809 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.655609 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.656830 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.657155 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.657237 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:07.658970 (+ 1733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:57:07.658977 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:57:07.658983 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:57:07.659013 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:57:07.660664 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:07.661123 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:07.661126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:07.661128 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:07.661206 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:07.661532 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:07.661535 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:07.661537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:07.661738 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:57:07.661841 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:07.662192 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 11:57:07.662321 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 11:57:07.666632 19024 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33144:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:57:07.671397 19024 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:57:07.704591 19568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:07.689094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:45685 (local address 127.0.0.1:42526)
0811 11:57:07.693792 (+ 4698us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:07.693840 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 11:57:07.693969 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:57:07.695366 (+ 1397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:57:07.695373 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:57:07.696087 (+ 714us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:57:07.696232 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:07.696240 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.697627 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.697630 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:07.700202 (+ 2572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:07.700207 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.704064 (+ 3857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.704068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:07.704172 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:07.704213 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 11:57:07.704312 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 11:57:07.704428 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4498,"mutex_wait_us":137,"thread_start_us":85,"threads_started":1}
I20260811 11:57:07.705446 19565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:07.690291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42526 (local address 127.6.143.131:45685)
0811 11:57:07.694705 (+ 4414us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:07.694718 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:07.694744 (+ 26us) server_negotiation.cc:407] Connection header received
0811 11:57:07.694822 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:07.694827 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:07.695040 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:07.695186 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:57:07.696371 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.697500 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.700318 (+ 2818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.701102 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.701207 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:07.701353 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:07.705182 (+ 3829us) server_negotiation.cc:299] Negotiation successful
0811 11:57:07.705304 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4306,"thread_start_us":105,"threads_started":1}
I20260811 11:57:07.708864 19477 tablet_service.cc:1467] Processing CreateTablet for tablet 9cc99def52604a36ada97a6da04723de (DEFAULT_TABLE table=test-table [id=33b897e93e284080ad0e895b8a7ac5dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:57:07.710074 19477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cc99def52604a36ada97a6da04723de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:57:07.714720 19564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:07.692562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:37721 (local address 127.0.0.1:51966)
0811 11:57:07.694403 (+ 1841us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:07.694413 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 11:57:07.694473 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:57:07.703673 (+ 9200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:57:07.703676 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:57:07.703691 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:57:07.703771 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:07.703776 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.705316 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.705319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:07.706410 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:07.706423 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.714327 (+ 7904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.714331 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:07.714427 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:07.714455 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 11:57:07.714510 (+ 55us) client_negotiation.cc:240] Negotiation successful
0811 11:57:07.714583 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1795,"thread_start_us":86,"threads_started":1}
I20260811 11:57:07.715379 19570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:07.692737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51966 (local address 127.6.143.130:37721)
0811 11:57:07.703235 (+ 10498us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:07.703251 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:07.703274 (+ 23us) server_negotiation.cc:407] Connection header received
0811 11:57:07.703346 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:07.703351 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:07.703524 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:07.703931 (+ 407us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:57:07.704085 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.705177 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.706598 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.708360 (+ 1762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.714930 (+ 6570us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:07.715089 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:07.715141 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 11:57:07.715245 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10376,"thread_start_us":60,"threads_started":1}
I20260811 11:57:07.718400 19314 tablet_service.cc:1467] Processing CreateTablet for tablet 9cc99def52604a36ada97a6da04723de (DEFAULT_TABLE table=test-table [id=33b897e93e284080ad0e895b8a7ac5dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:57:07.719506 19314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cc99def52604a36ada97a6da04723de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:57:07.733871 19566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:07.691748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:41355 (local address 127.0.0.1:53860)
0811 11:57:07.694176 (+ 2428us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:07.694187 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:57:07.694262 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:57:07.699948 (+ 5686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:57:07.699952 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:57:07.699971 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:57:07.700093 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:07.700100 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.702805 (+ 2705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.702808 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:07.703944 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:07.703953 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.733420 (+ 29467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.733425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:07.733532 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:07.733557 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 11:57:07.733618 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 11:57:07.733726 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2374,"mutex_wait_us":1,"thread_start_us":93,"threads_started":1}
I20260811 11:57:07.736152 19567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:07.692119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53860 (local address 127.6.143.129:41355)
0811 11:57:07.697350 (+ 5231us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:07.697357 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:07.697380 (+ 23us) server_negotiation.cc:407] Connection header received
0811 11:57:07.697451 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:07.697456 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:07.697660 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:07.701463 (+ 3803us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:57:07.701597 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.702665 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.732205 (+ 29540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.733090 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.733207 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:07.733350 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:07.735877 (+ 2527us) server_negotiation.cc:299] Negotiation successful
0811 11:57:07.736012 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5123,"thread_start_us":88,"threads_started":1}
I20260811 11:57:07.736269 19571 tablet_bootstrap.cc:492] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde: Bootstrap starting.
I20260811 11:57:07.739257 19571 tablet_bootstrap.cc:654] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde: Neither blocks nor log segments found. Creating new log.
I20260811 11:57:07.740381 19186 tablet_service.cc:1467] Processing CreateTablet for tablet 9cc99def52604a36ada97a6da04723de (DEFAULT_TABLE table=test-table [id=33b897e93e284080ad0e895b8a7ac5dc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:57:07.741478 19186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9cc99def52604a36ada97a6da04723de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:57:07.743790 19571 log.cc:824] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde: Log is configured to *not* fsync() on all Append() calls
I20260811 11:57:07.744590 19572 tablet_bootstrap.cc:492] T 9cc99def52604a36ada97a6da04723de P 3f3eaa68ead84ac78bea4928da898fa3: Bootstrap starting.
I20260811 11:57:07.751962 19574 tablet_bootstrap.cc:492] T 9cc99def52604a36ada97a6da04723de P db0f8ceec7b84b73accc75fab100f66b: Bootstrap starting.
I20260811 11:57:07.754992 19572 tablet_bootstrap.cc:654] T 9cc99def52604a36ada97a6da04723de P 3f3eaa68ead84ac78bea4928da898fa3: Neither blocks nor log segments found. Creating new log.
I20260811 11:57:07.756605 19574 tablet_bootstrap.cc:654] T 9cc99def52604a36ada97a6da04723de P db0f8ceec7b84b73accc75fab100f66b: Neither blocks nor log segments found. Creating new log.
I20260811 11:57:07.757488 19574 log.cc:824] T 9cc99def52604a36ada97a6da04723de P db0f8ceec7b84b73accc75fab100f66b: Log is configured to *not* fsync() on all Append() calls
I20260811 11:57:07.759214 19572 log.cc:824] T 9cc99def52604a36ada97a6da04723de P 3f3eaa68ead84ac78bea4928da898fa3: Log is configured to *not* fsync() on all Append() calls
I20260811 11:57:07.761889 19571 tablet_bootstrap.cc:492] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde: No bootstrap required, opened a new log
I20260811 11:57:07.762108 19571 ts_tablet_manager.cc:1397] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde: Time spent bootstrapping tablet: real 0.026s user 0.006s sys 0.000s
I20260811 11:57:07.766665 19571 raft_consensus.cc:348] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b0a55200ec74366862314af9ecb8bde" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 45685 } } peers { permanent_uuid: "3f3eaa68ead84ac78bea4928da898fa3" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37721 } } peers { permanent_uuid: "db0f8ceec7b84b73accc75fab100f66b" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41355 } }
I20260811 11:57:07.767127 19571 raft_consensus.cc:374] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:57:07.767235 19571 raft_consensus.cc:729] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b0a55200ec74366862314af9ecb8bde, State: Initialized, Role: FOLLOWER
I20260811 11:57:07.767952 19571 consensus_queue.cc:260] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b0a55200ec74366862314af9ecb8bde" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 45685 } } peers { permanent_uuid: "3f3eaa68ead84ac78bea4928da898fa3" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37721 } } peers { permanent_uuid: "db0f8ceec7b84b73accc75fab100f66b" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41355 } }
I20260811 11:57:07.769224 19571 ts_tablet_manager.cc:1428] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 11:57:07.769642 19550 heartbeater.cc:503] Master 127.6.143.190:37809 was elected leader, sending a full tablet report...
I20260811 11:57:07.775972 19574 tablet_bootstrap.cc:492] T 9cc99def52604a36ada97a6da04723de P db0f8ceec7b84b73accc75fab100f66b: No bootstrap required, opened a new log
I20260811 11:57:07.776242 19574 ts_tablet_manager.cc:1397] T 9cc99def52604a36ada97a6da04723de P db0f8ceec7b84b73accc75fab100f66b: Time spent bootstrapping tablet: real 0.025s user 0.005s sys 0.000s
I20260811 11:57:07.780807 19574 raft_consensus.cc:348] T 9cc99def52604a36ada97a6da04723de P db0f8ceec7b84b73accc75fab100f66b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b0a55200ec74366862314af9ecb8bde" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 45685 } } peers { permanent_uuid: "3f3eaa68ead84ac78bea4928da898fa3" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37721 } } peers { permanent_uuid: "db0f8ceec7b84b73accc75fab100f66b" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41355 } }
I20260811 11:57:07.781625 19574 raft_consensus.cc:374] T 9cc99def52604a36ada97a6da04723de P db0f8ceec7b84b73accc75fab100f66b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:57:07.781741 19574 raft_consensus.cc:729] T 9cc99def52604a36ada97a6da04723de P db0f8ceec7b84b73accc75fab100f66b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db0f8ceec7b84b73accc75fab100f66b, State: Initialized, Role: FOLLOWER
I20260811 11:57:07.783385 19572 tablet_bootstrap.cc:492] T 9cc99def52604a36ada97a6da04723de P 3f3eaa68ead84ac78bea4928da898fa3: No bootstrap required, opened a new log
I20260811 11:57:07.783636 19572 ts_tablet_manager.cc:1397] T 9cc99def52604a36ada97a6da04723de P 3f3eaa68ead84ac78bea4928da898fa3: Time spent bootstrapping tablet: real 0.039s user 0.004s sys 0.002s
I20260811 11:57:07.787633 19572 raft_consensus.cc:348] T 9cc99def52604a36ada97a6da04723de P 3f3eaa68ead84ac78bea4928da898fa3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b0a55200ec74366862314af9ecb8bde" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 45685 } } peers { permanent_uuid: "3f3eaa68ead84ac78bea4928da898fa3" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37721 } } peers { permanent_uuid: "db0f8ceec7b84b73accc75fab100f66b" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41355 } }
I20260811 11:57:07.788075 19572 raft_consensus.cc:374] T 9cc99def52604a36ada97a6da04723de P 3f3eaa68ead84ac78bea4928da898fa3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:57:07.788183 19572 raft_consensus.cc:729] T 9cc99def52604a36ada97a6da04723de P 3f3eaa68ead84ac78bea4928da898fa3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f3eaa68ead84ac78bea4928da898fa3, State: Initialized, Role: FOLLOWER
I20260811 11:57:07.789161 19574 consensus_queue.cc:260] T 9cc99def52604a36ada97a6da04723de P db0f8ceec7b84b73accc75fab100f66b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b0a55200ec74366862314af9ecb8bde" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 45685 } } peers { permanent_uuid: "3f3eaa68ead84ac78bea4928da898fa3" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37721 } } peers { permanent_uuid: "db0f8ceec7b84b73accc75fab100f66b" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41355 } }
I20260811 11:57:07.788761 19572 consensus_queue.cc:260] T 9cc99def52604a36ada97a6da04723de P 3f3eaa68ead84ac78bea4928da898fa3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b0a55200ec74366862314af9ecb8bde" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 45685 } } peers { permanent_uuid: "3f3eaa68ead84ac78bea4928da898fa3" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37721 } } peers { permanent_uuid: "db0f8ceec7b84b73accc75fab100f66b" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41355 } }
I20260811 11:57:07.790374 19572 ts_tablet_manager.cc:1428] T 9cc99def52604a36ada97a6da04723de P 3f3eaa68ead84ac78bea4928da898fa3: Time spent starting tablet: real 0.007s user 0.003s sys 0.002s
I20260811 11:57:07.791275 19574 ts_tablet_manager.cc:1428] T 9cc99def52604a36ada97a6da04723de P db0f8ceec7b84b73accc75fab100f66b: Time spent starting tablet: real 0.015s user 0.006s sys 0.000s
I20260811 11:57:07.791795 19402 heartbeater.cc:503] Master 127.6.143.190:37809 was elected leader, sending a full tablet report...
I20260811 11:57:07.794931 19257 heartbeater.cc:503] Master 127.6.143.190:37809 was elected leader, sending a full tablet report...
Aug 11 11:57:07 dist-test-slave-zkpd krb5kdc[18498](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449426, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for HTTP/127.6.143.148@KRBTEST.COM
I20260811 11:57:07.878968 19579 raft_consensus.cc:482] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260811 11:57:07.879074 19551 tablet.cc:2369] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:57:07.879237 19579 raft_consensus.cc:504] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b0a55200ec74366862314af9ecb8bde" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 45685 } } peers { permanent_uuid: "3f3eaa68ead84ac78bea4928da898fa3" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37721 } } peers { permanent_uuid: "db0f8ceec7b84b73accc75fab100f66b" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41355 } }
I20260811 11:57:07.880674 19579 leader_election.cc:288] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3f3eaa68ead84ac78bea4928da898fa3 (127.6.143.130:37721), db0f8ceec7b84b73accc75fab100f66b (127.6.143.129:41355)
I20260811 11:57:07.885968 19552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:07.881114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:41355 (local address 127.6.143.131:40389)
0811 11:57:07.881265 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:07.881284 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 11:57:07.881394 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:57:07.882116 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:57:07.882120 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:57:07.882140 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:57:07.882262 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:07.882269 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.883269 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.883272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:07.884583 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:07.884607 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.885553 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.885557 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:07.885658 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:07.885681 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 11:57:07.885762 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 11:57:07.885835 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 11:57:07.886879 19567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:07.881419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:40389 (local address 127.6.143.129:41355)
0811 11:57:07.881557 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:07.881561 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:07.881574 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:57:07.881629 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:07.881631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:07.881680 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:07.881763 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:57:07.882395 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.883160 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.884746 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.885326 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.885443 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:07.885494 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:07.886677 (+ 1183us) server_negotiation.cc:299] Negotiation successful
0811 11:57:07.886765 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 11:57:07.887480 19188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cc99def52604a36ada97a6da04723de" candidate_uuid: "4b0a55200ec74366862314af9ecb8bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db0f8ceec7b84b73accc75fab100f66b" is_pre_election: true
I20260811 11:57:07.887835 19188 raft_consensus.cc:2454] T 9cc99def52604a36ada97a6da04723de P db0f8ceec7b84b73accc75fab100f66b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b0a55200ec74366862314af9ecb8bde in term 0.
I20260811 11:57:07.888413 19427 leader_election.cc:302] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4b0a55200ec74366862314af9ecb8bde, db0f8ceec7b84b73accc75fab100f66b; no voters:
I20260811 11:57:07.888939 19579 raft_consensus.cc:2781] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:57:07.889022 19579 raft_consensus.cc:482] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:57:07.889061 19579 raft_consensus.cc:3037] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:57:07.890401 19579 raft_consensus.cc:504] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b0a55200ec74366862314af9ecb8bde" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 45685 } } peers { permanent_uuid: "3f3eaa68ead84ac78bea4928da898fa3" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37721 } } peers { permanent_uuid: "db0f8ceec7b84b73accc75fab100f66b" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41355 } }
I20260811 11:57:07.890549 19570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:07.882021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:50331 (local address 127.6.143.130:37721)
0811 11:57:07.882168 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:07.882173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:07.886389 (+ 4216us) server_negotiation.cc:407] Connection header received
0811 11:57:07.886448 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:07.886451 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:07.886500 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:07.886570 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:57:07.886885 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.887621 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.889079 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.889676 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.889769 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:07.889815 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:07.890331 (+ 516us) server_negotiation.cc:299] Negotiation successful
0811 11:57:07.890437 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 11:57:07.890759 19579 leader_election.cc:288] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde [CANDIDATE]: Term 1 election: Requested vote from peers 3f3eaa68ead84ac78bea4928da898fa3 (127.6.143.130:37721), db0f8ceec7b84b73accc75fab100f66b (127.6.143.129:41355)
I20260811 11:57:07.890779 19552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:07.881524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:37721 (local address 127.6.143.131:50331)
0811 11:57:07.886123 (+ 4599us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:07.886137 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:57:07.886210 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:57:07.886616 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:57:07.886620 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:57:07.886643 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:57:07.886724 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:07.886730 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.887761 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.887765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:07.888928 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:07.888940 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.889921 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.889925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:07.890012 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:07.890033 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 11:57:07.890096 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 11:57:07.890162 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4548,"thread_start_us":131,"threads_started":1}
I20260811 11:57:07.891407 19353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cc99def52604a36ada97a6da04723de" candidate_uuid: "4b0a55200ec74366862314af9ecb8bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f3eaa68ead84ac78bea4928da898fa3" is_pre_election: true
I20260811 11:57:07.891450 19350 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cc99def52604a36ada97a6da04723de" candidate_uuid: "4b0a55200ec74366862314af9ecb8bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3f3eaa68ead84ac78bea4928da898fa3"
I20260811 11:57:07.891436 19188 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9cc99def52604a36ada97a6da04723de" candidate_uuid: "4b0a55200ec74366862314af9ecb8bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db0f8ceec7b84b73accc75fab100f66b"
I20260811 11:57:07.891566 19188 raft_consensus.cc:3037] T 9cc99def52604a36ada97a6da04723de P db0f8ceec7b84b73accc75fab100f66b [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:57:07.891680 19353 raft_consensus.cc:2454] T 9cc99def52604a36ada97a6da04723de P 3f3eaa68ead84ac78bea4928da898fa3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4b0a55200ec74366862314af9ecb8bde in term 0.
I20260811 11:57:07.892920 19188 raft_consensus.cc:2454] T 9cc99def52604a36ada97a6da04723de P db0f8ceec7b84b73accc75fab100f66b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4b0a55200ec74366862314af9ecb8bde in term 1.
I20260811 11:57:07.893299 19427 leader_election.cc:302] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 4b0a55200ec74366862314af9ecb8bde, db0f8ceec7b84b73accc75fab100f66b; no voters: 3f3eaa68ead84ac78bea4928da898fa3
I20260811 11:57:07.893471 19579 raft_consensus.cc:2781] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:57:07.893746 19579 raft_consensus.cc:686] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde [term 1 LEADER]: Becoming Leader. State: Replica: 4b0a55200ec74366862314af9ecb8bde, State: Running, Role: LEADER
I20260811 11:57:07.894083 19579 consensus_queue.cc:237] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b0a55200ec74366862314af9ecb8bde" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 45685 } } peers { permanent_uuid: "3f3eaa68ead84ac78bea4928da898fa3" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37721 } } peers { permanent_uuid: "db0f8ceec7b84b73accc75fab100f66b" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41355 } }
W20260811 11:57:07.897691 19258 tablet.cc:2369] T 9cc99def52604a36ada97a6da04723de P db0f8ceec7b84b73accc75fab100f66b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:57:07.901528 19023 catalog_manager.cc:5495] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde reported cstate change: term changed from 0 to 1, leader changed from <none> to 4b0a55200ec74366862314af9ecb8bde (127.6.143.131). New cstate: current_term: 1 leader_uuid: "4b0a55200ec74366862314af9ecb8bde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4b0a55200ec74366862314af9ecb8bde" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 45685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3f3eaa68ead84ac78bea4928da898fa3" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "db0f8ceec7b84b73accc75fab100f66b" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41355 } health_report { overall_health: UNKNOWN } } }
I20260811 11:57:07.954046 19565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:07.948732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42528 (local address 127.6.143.131:45685)
0811 11:57:07.948890 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:07.948896 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:07.949100 (+ 204us) server_negotiation.cc:407] Connection header received
0811 11:57:07.949205 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:07.949208 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:07.949294 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:07.949406 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:57:07.949824 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.950670 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.951920 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:07.952246 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:07.952344 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:07.952772 (+ 428us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:57:07.953359 (+ 587us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:57:07.953476 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:07.953822 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 11:57:07.953884 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260811 11:57:07.964794 19403 tablet.cc:2369] T 9cc99def52604a36ada97a6da04723de P 3f3eaa68ead84ac78bea4928da898fa3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:57:07.966331 19188 raft_consensus.cc:1261] T 9cc99def52604a36ada97a6da04723de P db0f8ceec7b84b73accc75fab100f66b [term 1 FOLLOWER]: Refusing update from remote peer 4b0a55200ec74366862314af9ecb8bde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:57:07.967255 19588 consensus_queue.cc:1035] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "db0f8ceec7b84b73accc75fab100f66b" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41355 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:57:07.972026 19350 raft_consensus.cc:3037] T 9cc99def52604a36ada97a6da04723de P 3f3eaa68ead84ac78bea4928da898fa3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:57:07.973729 19350 raft_consensus.cc:1261] T 9cc99def52604a36ada97a6da04723de P 3f3eaa68ead84ac78bea4928da898fa3 [term 1 FOLLOWER]: Refusing update from remote peer 4b0a55200ec74366862314af9ecb8bde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:57:07.974435 19588 consensus_queue.cc:1035] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "3f3eaa68ead84ac78bea4928da898fa3" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 37721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:57:07.979341 19591 mvcc.cc:205] Tried to move back new op lower bound from 7317296856934109184 to 7317296856656535552. Current Snapshot: MvccSnapshot[applied={T|T < 7317296856934109184}]
I20260811 11:57:07.979840 19592 mvcc.cc:205] Tried to move back new op lower bound from 7317296856934109184 to 7317296856656535552. Current Snapshot: MvccSnapshot[applied={T|T < 7317296856934109184}]
I20260811 11:57:07.988634 19595 mvcc.cc:205] Tried to move back new op lower bound from 7317296856934109184 to 7317296856656535552. Current Snapshot: MvccSnapshot[applied={T|T < 7317296856934109184}]
I20260811 11:57:07.992594 19024 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:57:07.992851 19024 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33144:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:57:07.995664 19024 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c4d31edfc65448ef82fd92d82046a11b: Sending DeleteTablet for 3 replicas of tablet 9cc99def52604a36ada97a6da04723de
I20260811 11:57:07.996591 19186 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cc99def52604a36ada97a6da04723de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:57:07 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:53860
I20260811 11:57:07.997670 19477 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cc99def52604a36ada97a6da04723de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:57:07 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:42526
I20260811 11:57:07.998190 19314 tablet_service.cc:1514] Processing DeleteTablet for tablet 9cc99def52604a36ada97a6da04723de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:57:07 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:51966
I20260811 11:57:07.999425 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 19114
I20260811 11:57:08.005416 19604 tablet_replica.cc:330] T 9cc99def52604a36ada97a6da04723de P 3f3eaa68ead84ac78bea4928da898fa3: stopping tablet replica
I20260811 11:57:08.005658 19603 tablet_replica.cc:330] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde: stopping tablet replica
I20260811 11:57:08.006084 19603 raft_consensus.cc:2229] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:57:08.006458 19604 raft_consensus.cc:2229] T 9cc99def52604a36ada97a6da04723de P 3f3eaa68ead84ac78bea4928da898fa3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:57:08.008280 19604 raft_consensus.cc:2258] T 9cc99def52604a36ada97a6da04723de P 3f3eaa68ead84ac78bea4928da898fa3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:57:08.008723 19603 raft_consensus.cc:2258] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:57:08.010656 19603 ts_tablet_manager.cc:1905] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:57:08.016184 19604 ts_tablet_manager.cc:1905] T 9cc99def52604a36ada97a6da04723de P 3f3eaa68ead84ac78bea4928da898fa3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:57:08.019307 19603 ts_tablet_manager.cc:1918] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:57:08.022110 19604 ts_tablet_manager.cc:1918] T 9cc99def52604a36ada97a6da04723de P 3f3eaa68ead84ac78bea4928da898fa3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:57:08.027388 19603 log.cc:1196] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-2/wal/wals/9cc99def52604a36ada97a6da04723de
I20260811 11:57:08.027596 19604 log.cc:1196] T 9cc99def52604a36ada97a6da04723de P 3f3eaa68ead84ac78bea4928da898fa3: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786449319379472-6718-0/minicluster-data/ts-1/wal/wals/9cc99def52604a36ada97a6da04723de
I20260811 11:57:08.028199 19603 ts_tablet_manager.cc:1939] T 9cc99def52604a36ada97a6da04723de P 4b0a55200ec74366862314af9ecb8bde: Deleting consensus metadata
I20260811 11:57:08.028613 19604 ts_tablet_manager.cc:1939] T 9cc99def52604a36ada97a6da04723de P 3f3eaa68ead84ac78bea4928da898fa3: Deleting consensus metadata
I20260811 11:57:08.030310 19011 catalog_manager.cc:4841] TS 3f3eaa68ead84ac78bea4928da898fa3 (127.6.143.130:37721): tablet 9cc99def52604a36ada97a6da04723de (table test-table [id=33b897e93e284080ad0e895b8a7ac5dc]) successfully deleted
I20260811 11:57:08.031808 19011 catalog_manager.cc:4841] TS 4b0a55200ec74366862314af9ecb8bde (127.6.143.131:45685): tablet 9cc99def52604a36ada97a6da04723de (table test-table [id=33b897e93e284080ad0e895b8a7ac5dc]) successfully deleted
W20260811 11:57:08.032554 19010 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.143.129:41355 (error 108)
I20260811 11:57:08.032871 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 19263
I20260811 11:57:08.043536 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 19407
I20260811 11:57:08.045542 19566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:08.045109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:41355 (local address 127.0.0.1:53866)
0811 11:57:08.045351 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:08.045430 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.129:41355: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137}
W20260811 11:57:08.045849 19010 catalog_manager.cc:4584] TS db0f8ceec7b84b73accc75fab100f66b (127.6.143.129:41355): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9cc99def52604a36ada97a6da04723de: Network error: Client connection negotiation failed: client connection to 127.6.143.129:41355: connect: Connection refused (error 111)
I20260811 11:57:08.054881 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 18992
I20260811 11:57:08.066984 6718 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 11:57:08.411339 6718 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 11:57:08.411479 6718 mini_ranger.cc:66] Stopping Ranger...
I20260811 11:57:08.756253 6718 mini_ranger.cc:68] Stopped Ranger
2026-08-11 11:57:08.756 UTC [18529] LOG: received smart shutdown request
2026-08-11 11:57:08.759 UTC [18529] LOG: background worker "logical replication launcher" (PID 18538) exited with exit code 1
2026-08-11 11:57:08.760 UTC [18533] LOG: shutting down
2026-08-11 11:57:08.909 UTC [18529] LOG: database system is shut down
2026-08-11T11:57:08Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (46286 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:57:09 dist-test-slave-zkpd krb5kdc[19615](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:57:09 dist-test-slave-zkpd krb5kdc[19615](info): set up 2 sockets
Aug 11 11:57:09 dist-test-slave-zkpd krb5kdc[19615](info): commencing operation
krb5kdc: starting...
W20260811 11:57:10.136943 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:57:10 dist-test-slave-zkpd krb5kdc[19615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449430, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:57:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:57:10Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.6.143.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.6.143.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.6.143.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/rangeradmin_127.6.143.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.6.143.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/rangeradmin_127.6.143.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.6.143.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.6.143.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.6.143.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/rangerlookup_127.6.143.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.6.143.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/rangerlookup_127.6.143.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.143.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.6.143.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.143.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/HTTP_127.6.143.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.143.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/HTTP_127.6.143.148@KRBTEST.COM.keytab.
I20260811 11:57:10.303607 6718 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskEMvzen/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/postgres -l logfile start
2026-08-11 11:57:12.568 UTC [19649] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 11:57:12.568 UTC [19649] LOG: listening on IPv4 address "127.6.143.148", port 40413
2026-08-11 11:57:12.571 UTC [19649] LOG: listening on Unix socket "/tmp/.s.PGSQL.40413"
2026-08-11 11:57:12.592 UTC [19652] LOG: database system was shut down at 2026-08-11 11:57:11 UTC
2026-08-11 11:57:12.595 UTC [19649] LOG: database system is ready to accept connections
I20260811 11:57:13.608561 6718 mini_postgres.cc:96] Postgres bound to 40413
2026-08-11 11:57:13.616 UTC [19661] FATAL: database "slave" does not exist
127.6.143.148:40413 - accepting connections
I20260811 11:57:13.616698 6718 mini_ranger.cc:161] Starting Ranger...
I20260811 11:57:13.631816 6718 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 11:57:14.046777 6718 mini_ranger.cc:87] Created ranger Postgres database
I20260811 11:57:14.046866 6718 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ranger-admin
2026-08-11 11:57:14,315 [I] DB FLAVOR :POSTGRES
2026-08-11 11:57:14,316 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 11:57:14,316 [I] Checking connection..
2026-08-11 11:57:14,316 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:40413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 11:57:14,562 [I] Checking connection passed.
2026-08-11 11:57:14,562 [I] --------- Verifying version history table ---------
2026-08-11 11:57:14,562 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:40413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 11:57:14,810 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 11:57:14,811 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:40413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 11:57:15,066 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 11:57:15,066 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 11:57:15,066 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:40413/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 11:57:15.327 UTC [19733] WARNING: there is no transaction in progress
2026-08-11 11:57:15,333 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 11:57:15,334 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:40413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 11:57:15,591 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 11:57:15,592 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 11:57:15,592 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:40413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 11:57:15,829 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:40413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 11:57:16,065 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:40413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-zkpd.c.gcp-upstream.internal','N') ;"
2026-08-11 11:57:16,304 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:40413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 11:57:16,563 [I] Table x_portal_user does not exist in database ranger
2026-08-11 11:57:16,563 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:40413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 11:57:16,825 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 11:57:16,825 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:40413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 11:57:17,050 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:40413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 11:57:17,301 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 11:57:17,301 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:40413/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 11:57:18.229 UTC [19849] WARNING: there is no transaction in progress
2026-08-11 11:57:18.246 UTC [19849] WARNING: there is no transaction in progress
2026-08-11 11:57:18.262 UTC [19849] WARNING: there is no transaction in progress
2026-08-11 11:57:18.276 UTC [19849] WARNING: there is no transaction in progress
2026-08-11 11:57:18.291 UTC [19849] WARNING: there is no transaction in progress
2026-08-11 11:57:18.310 UTC [19849] WARNING: there is no transaction in progress
2026-08-11 11:57:18.363 UTC [19849] WARNING: there is no transaction in progress
2026-08-11 11:57:18.369 UTC [19849] WARNING: there is no transaction in progress
2026-08-11 11:57:18.383 UTC [19849] WARNING: there is no transaction in progress
2026-08-11 11:57:18.394 UTC [19849] WARNING: there is no transaction in progress
2026-08-11 11:57:18.404 UTC [19849] WARNING: there is no transaction in progress
2026-08-11 11:57:18.418 UTC [19849] WARNING: there is no transaction in progress
2026-08-11 11:57:18.428 UTC [19849] WARNING: there is no transaction in progress
2026-08-11 11:57:18.434 UTC [19849] WARNING: there is no transaction in progress
2026-08-11 11:57:18.440 UTC [19849] WARNING: there is no transaction in progress
2026-08-11 11:57:18.446 UTC [19849] WARNING: there is no transaction in progress
2026-08-11 11:57:19,049 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 11:57:19,049 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:40413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 11:57:19,303 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 11:57:19,304 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:40413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 11:57:19,536 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 11:57:19,537 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:40413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 11:57:19,774 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 11:57:19,774 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:40413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 11:57:20,017 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:40413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 11:57:20,267 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:40413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-zkpd.c.gcp-upstream.internal';"
2026-08-11 11:57:20,489 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 11:57:20,489 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:40413/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 11:57:20,741 [I] DB_PATCHES have already been applied
I20260811 11:57:20.747248 6718 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ranger-admin:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskEMvzen/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 11:57:20.747347 6718 mini_ranger.cc:193] Using host: 127.6.143.148
I20260811 11:57:20.751763 6718 mini_ranger.cc:227] Ranger admin URL: http://127.6.143.148:47601
Aug 11, 2026 11:57:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 11:57:21 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 11:57:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 11:57:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 11:57:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 11:57:21 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 11:57:21 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 11:57:21.918536 6718 mini_ranger.cc:160] Time spent starting Ranger: real 8.302s user 0.000s sys 0.011s
Aug 11, 2026 11:57:41 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 11:57:41 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 11, 2026 11:57:41 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 11:57:41 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 11:57:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 11:57:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 11:57:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 11:57:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 11:57:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 11:57:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 11:57:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 11:57:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 11:57:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 11:57:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 11:57:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 11:57:41 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 11:57:41 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 11:57:42 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 11:57:42.973932 6718 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.6.143.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.6.143.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.6.143.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/rangerkms_127.6.143.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.6.143.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/rangerkms_127.6.143.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.143.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.6.143.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.143.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/HTTP_127.6.143.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.143.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/HTTP_127.6.143.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 11:57:43 dist-test-slave-zkpd krb5kdc[19615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449463, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 11:57:43.059334 6718 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 11:57:43.069072 6718 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 11:57:43.879060 6718 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 11:57:43.879154 6718 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ranger-kms
I20260811 11:57:43.879177 6718 mini_ranger_kms.cc:227] Using postgres at 127.6.143.148:40413
2026-08-11 11:57:44,056 [I] DB FLAVOR :POSTGRES
2026-08-11 11:57:44,057 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 11:57:44,057 [I] Checking connection
2026-08-11 11:57:44,057 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:40413/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 11:57:44,293 [I] connection success
2026-08-11 11:57:44,294 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 11:57:44,294 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 11:57:44,294 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:40413/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 11:57:44,529 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 11:57:44,529 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 11:57:44,529 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 11:57:44,529 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.143.148:40413/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 11:57:44,790 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 11:57:45.231963 6718 mini_ranger_kms.cc:310] Created kms service
I20260811 11:57:45.446575 6718 mini_ranger_kms.cc:326] Created kudu user
I20260811 11:57:45.494338 6718 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 11:57:45 dist-test-slave-zkpd krb5kdc[19615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449463, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.6.143.148@KRBTEST.COM
I20260811 11:57:45.894279 6718 mini_ranger_kms.cc:384] Added ranger policy
I20260811 11:57:45.894550 6718 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ranger-kms:/tmp/dist-test-taskEMvzen/build/debug/bin/postgresql.jar:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskEMvzen/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 11:57:45.894580 6718 mini_ranger_kms.cc:240] Using host: 127.6.143.148
I20260811 11:57:45.899489 6718 mini_ranger_kms.cc:276] Ranger KMS PID: 20047
I20260811 11:57:45.899578 6718 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.6.143.148:42401
Aug 11, 2026 11:57:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 11:57:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 11:57:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 11:57:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 11:57:47 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 11:57:47 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 11:57:47 dist-test-slave-zkpd krb5kdc[19615](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786449467, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.143.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:57:48.287755 6718 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.228s user 0.002s sys 0.161s
I20260811 11:57:48.287918 6718 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 11:57:48.288015 6718 mini_ranger_kms.cc:401] 127.6.143.148:42401/kms/v1/keys
Aug 11 11:57:48 dist-test-slave-zkpd krb5kdc[19615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449467, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.143.148@KRBTEST.COM for HTTP/127.6.143.148@KRBTEST.COM
Aug 11 11:57:48 dist-test-slave-zkpd krb5kdc[19615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449467, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.143.148@KRBTEST.COM for HTTP/127.6.143.148@KRBTEST.COM
Aug 11 11:57:51 dist-test-slave-zkpd krb5kdc[19615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
I20260811 11:57:51.050791 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:39355
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:45061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:39355
--ranger_config_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.143.148:42401/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:57:51.164685 20101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:57:51.165009 20101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:57:51.165095 20101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:57:51.169126 20101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:57:51.169178 20101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:57:51.169198 20101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:57:51.169214 20101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:57:51.169229 20101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:57:51.174680 20101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:45061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/wal
--ranger_kms_url=127.6.143.148:42401/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:39355
--ranger_config_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:39355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:57:51.175733 20101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:57:51.176867 20101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 11:57:51.186709 20101 server_base.cc:1034] running on GCE node
W20260811 11:57:51.186623 20108 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:57:51.186651 20106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:57:51.186933 20110 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:57:51.187759 20101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:57:51.189178 20101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:57:51.190384 20101 hybrid_clock.cc:648] HybridClock initialized: now 1786449471190356 us; error 50 us; skew 500 ppm
Aug 11 11:57:51 dist-test-slave-zkpd krb5kdc[19615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449471, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:57:51.193858 20101 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:57:51.194952 20101 webserver.cc:460] Webserver started at http://127.6.143.190:46475/ using document root <none> and password file <none>
I20260811 11:57:51.195631 20101 fs_manager.cc:359] Metadata directory not provided
I20260811 11:57:51.195703 20101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:57:51.195958 20101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 11:57:51 dist-test-slave-zkpd krb5kdc[19615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449471, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for HTTP/127.6.143.148@KRBTEST.COM
I20260811 11:57:51.253667 20101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "e25823411ce8459c8fa98e4f8d7b4174"
format_stamp: "Formatted at 2026-08-11 11:57:51 on dist-test-slave-zkpd"
server_key: "8cdf99a4e5e4b3df1d07e812baad2f04"
server_key_iv: "4bd073ca3ee67c4176fdf3bd66c5b506"
server_key_version: "kuduclusterkey@0"
I20260811 11:57:51.254437 20101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "e25823411ce8459c8fa98e4f8d7b4174"
format_stamp: "Formatted at 2026-08-11 11:57:51 on dist-test-slave-zkpd"
server_key: "8cdf99a4e5e4b3df1d07e812baad2f04"
server_key_iv: "4bd073ca3ee67c4176fdf3bd66c5b506"
server_key_version: "kuduclusterkey@0"
I20260811 11:57:51.259572 20101 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 11:57:51.277272 20118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:51.278936 20101 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 11:57:51.279078 20101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "e25823411ce8459c8fa98e4f8d7b4174"
format_stamp: "Formatted at 2026-08-11 11:57:51 on dist-test-slave-zkpd"
server_key: "8cdf99a4e5e4b3df1d07e812baad2f04"
server_key_iv: "4bd073ca3ee67c4176fdf3bd66c5b506"
server_key_version: "kuduclusterkey@0"
I20260811 11:57:51.279209 20101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:57:51.307267 20101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:57:51.310861 20101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:57:51.311074 20101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:57:51.319772 20101 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:39355
I20260811 11:57:51.319802 20177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:39355 every 8 connection(s)
I20260811 11:57:51.320943 20101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:57:51.323885 20178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:57:51.329030 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 20101
I20260811 11:57:51.329151 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:57:51.329443 6718 external_mini_cluster.cc:1428] Setting key a6f5b38ecfce99f5372dc2389087052e
I20260811 11:57:51.329639 20178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174: Bootstrap starting.
I20260811 11:57:51.332533 20178 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174: Neither blocks nor log segments found. Creating new log.
I20260811 11:57:51.333431 20178 log.cc:824] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174: Log is configured to *not* fsync() on all Append() calls
I20260811 11:57:51.335304 20178 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174: No bootstrap required, opened a new log
Aug 11 11:57:51 dist-test-slave-zkpd krb5kdc[19615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:57:51.340039 20178 raft_consensus.cc:348] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e25823411ce8459c8fa98e4f8d7b4174" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39355 } }
I20260811 11:57:51.340274 20178 raft_consensus.cc:374] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:57:51.340328 20178 raft_consensus.cc:729] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e25823411ce8459c8fa98e4f8d7b4174, State: Initialized, Role: FOLLOWER
I20260811 11:57:51.340785 20178 consensus_queue.cc:260] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e25823411ce8459c8fa98e4f8d7b4174" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39355 } }
I20260811 11:57:51.340983 20178 raft_consensus.cc:388] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:57:51.341059 20178 raft_consensus.cc:482] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:57:51.341171 20178 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:57:51.342177 20178 raft_consensus.cc:504] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e25823411ce8459c8fa98e4f8d7b4174" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39355 } }
I20260811 11:57:51.342531 20178 leader_election.cc:302] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e25823411ce8459c8fa98e4f8d7b4174; no voters:
I20260811 11:57:51.342921 20178 leader_election.cc:288] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:57:51.342980 20183 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:57:51.343524 20183 raft_consensus.cc:686] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [term 1 LEADER]: Becoming Leader. State: Replica: e25823411ce8459c8fa98e4f8d7b4174, State: Running, Role: LEADER
I20260811 11:57:51.343825 20183 consensus_queue.cc:237] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e25823411ce8459c8fa98e4f8d7b4174" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39355 } }
I20260811 11:57:51.344319 20178 sys_catalog.cc:564] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:57:51.345045 20183 sys_catalog.cc:455] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e25823411ce8459c8fa98e4f8d7b4174. Latest consensus state: current_term: 1 leader_uuid: "e25823411ce8459c8fa98e4f8d7b4174" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e25823411ce8459c8fa98e4f8d7b4174" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39355 } } }
I20260811 11:57:51.345149 20183 sys_catalog.cc:458] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [sys.catalog]: This master's current role is: LEADER
I20260811 11:57:51.345333 20183 sys_catalog.cc:455] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e25823411ce8459c8fa98e4f8d7b4174" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e25823411ce8459c8fa98e4f8d7b4174" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39355 } } }
I20260811 11:57:51.345395 20183 sys_catalog.cc:458] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [sys.catalog]: This master's current role is: LEADER
I20260811 11:57:51.345651 20192 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:57:51.346681 20181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:51.330836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55826 (local address 127.6.143.190:39355)
0811 11:57:51.331312 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:51.331330 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:51.331383 (+ 53us) server_negotiation.cc:407] Connection header received
0811 11:57:51.332203 (+ 820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:51.332228 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:51.332667 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:51.333046 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:57:51.333946 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:51.335395 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:51.336625 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:51.337306 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:51.337474 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:51.340516 (+ 3042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:57:51.340522 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:57:51.340540 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:57:51.340569 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:57:51.342619 (+ 2050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:51.343182 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:51.343188 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:51.343194 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:51.343318 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:51.343651 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:51.343654 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:51.343655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:51.344167 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:57:51.344350 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:51.345293 (+ 943us) server_negotiation.cc:299] Negotiation successful
0811 11:57:51.345609 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":140,"threads_started":1}
I20260811 11:57:51.348855 20192 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:57:51.349269 20178 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 11:57:51.362771 20192 catalog_manager.cc:1347] Generated new cluster ID: 3d18fedeba17477e9bd39d354cfb0b22
I20260811 11:57:51.362851 20192 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:57:51.377471 20192 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:57:51.378788 20192 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:57:51.386868 20192 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174: Generated new TSK 0
I20260811 11:57:51.388360 20192 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
I20260811 11:57:52.175474 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.143.148:42401/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:0
--local_ip_for_outbound_sockets=127.6.143.129
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:39355
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:45061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:57:52.348728 20227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:57:52.349176 20227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:57:52.349328 20227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:57:52.355609 20227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:57:52.355688 20227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:57:52.355789 20227 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:57:52.363667 20227 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:45061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.6.143.148:42401/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:39355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:57:52.365108 20227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:57:52.366616 20227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:57:52.379386 20235 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:57:52.379381 20234 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:57:52.379588 20227 server_base.cc:1034] running on GCE node
W20260811 11:57:52.379383 20237 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:57:52.380266 20227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:57:52.381057 20227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:57:52.382272 20227 hybrid_clock.cc:648] HybridClock initialized: now 1786449472382247 us; error 35 us; skew 500 ppm
Aug 11 11:57:52 dist-test-slave-zkpd krb5kdc[19615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449472, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:57:52.385916 20227 init.cc:377] Logged in from keytab as kudu/127.6.143.129@KRBTEST.COM (short username kudu)
I20260811 11:57:52.386744 20227 webserver.cc:460] Webserver started at http://127.6.143.129:34615/ using document root <none> and password file <none>
I20260811 11:57:52.387423 20227 fs_manager.cc:359] Metadata directory not provided
I20260811 11:57:52.387521 20227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:57:52.387751 20227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 11:57:52 dist-test-slave-zkpd krb5kdc[19615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449472, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for HTTP/127.6.143.148@KRBTEST.COM
I20260811 11:57:52.409473 20227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/data/instance:
uuid: "24989c5801fc4f4d8b2a2c3c20515386"
format_stamp: "Formatted at 2026-08-11 11:57:52 on dist-test-slave-zkpd"
server_key: "066560adb1dbe89b5c64d257fd4f2205"
server_key_iv: "413cc37ea248184a1dd18996541b5de8"
server_key_version: "kuduclusterkey@0"
I20260811 11:57:52.410109 20227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance:
uuid: "24989c5801fc4f4d8b2a2c3c20515386"
format_stamp: "Formatted at 2026-08-11 11:57:52 on dist-test-slave-zkpd"
server_key: "066560adb1dbe89b5c64d257fd4f2205"
server_key_iv: "413cc37ea248184a1dd18996541b5de8"
server_key_version: "kuduclusterkey@0"
I20260811 11:57:52.415709 20227 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 11:57:52.437141 20244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:52.439064 20227 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260811 11:57:52.439211 20227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "24989c5801fc4f4d8b2a2c3c20515386"
format_stamp: "Formatted at 2026-08-11 11:57:52 on dist-test-slave-zkpd"
server_key: "066560adb1dbe89b5c64d257fd4f2205"
server_key_iv: "413cc37ea248184a1dd18996541b5de8"
server_key_version: "kuduclusterkey@0"
I20260811 11:57:52.439370 20227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:57:52.478287 20227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:57:52.482522 20227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:57:52.482790 20227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:57:52.483671 20227 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:57:52.484922 20227 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:57:52.485026 20227 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:52.485155 20227 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:57:52.485234 20227 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:52.499083 20227 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:38515
I20260811 11:57:52.500384 20227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:57:52.501221 20372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:38515 every 8 connection(s)
I20260811 11:57:52.507434 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 20227
I20260811 11:57:52.507558 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance
I20260811 11:57:52.507879 6718 external_mini_cluster.cc:1428] Setting key 2c4f4a879bf1c2b1764ef87dd765082f
WARNING: no policy specified for kudu/127.6.143.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.130@KRBTEST.COM" created.
Aug 11 11:57:52 dist-test-slave-zkpd krb5kdc[19615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449472, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:57:52.537101 20376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:52.505170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:56335 (local address 127.6.143.190:39355)
0811 11:57:52.505630 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:52.505636 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:52.512428 (+ 6792us) server_negotiation.cc:407] Connection header received
0811 11:57:52.513796 (+ 1368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:52.513800 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:52.513859 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:52.513924 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:57:52.515679 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:52.516535 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:52.518298 (+ 1763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:52.518655 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:52.518758 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:52.526164 (+ 7406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:57:52.526171 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:57:52.526173 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:57:52.526203 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:57:52.528250 (+ 2047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:52.528912 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:52.528919 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:52.528923 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:52.529020 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:52.529483 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:52.529488 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:52.529490 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:52.529745 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:57:52.529844 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:52.536706 (+ 6862us) server_negotiation.cc:299] Negotiation successful
0811 11:57:52.536909 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":87,"threads_started":1}
I20260811 11:57:52.538169 20375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:52.504026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39355 (local address 127.6.143.129:56335)
0811 11:57:52.512262 (+ 8236us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:52.512325 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 11:57:52.513477 (+ 1152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:57:52.514375 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:57:52.514385 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:57:52.514777 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:57:52.515484 (+ 707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:52.515506 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:52.516719 (+ 1213us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:52.516723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:52.518142 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:52.518155 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:52.518982 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:52.518986 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:52.519185 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:52.524103 (+ 4918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:57:52.524130 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:57:52.525942 (+ 1812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:57:52.528403 (+ 2461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:52.528410 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:52.528427 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:52.528782 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:52.529163 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:52.529166 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:52.529168 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:52.529359 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:52.535990 (+ 6631us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:57:52.535998 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:57:52.536366 (+ 368us) client_negotiation.cc:769] Sending connection context
0811 11:57:52.536626 (+ 260us) client_negotiation.cc:240] Negotiation successful
0811 11:57:52.537026 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7972,"thread_start_us":135,"threads_started":1}
I20260811 11:57:52.539871 20373 heartbeater.cc:347] Connected to a master server at 127.6.143.190:39355
I20260811 11:57:52.540169 20373 heartbeater.cc:464] Registering TS with master...
I20260811 11:57:52.540802 20373 heartbeater.cc:511] Master 127.6.143.190:39355 requested a full tablet report, sending...
I20260811 11:57:52.542690 20137 ts_manager.cc:194] Registered new tserver with Master: 24989c5801fc4f4d8b2a2c3c20515386 (127.6.143.129:38515)
I20260811 11:57:52.544246 20137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.129@KRBTEST.COM'} at 127.6.143.129:56335
Aug 11 11:57:52 dist-test-slave-zkpd krb5kdc[19615](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786449472, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
I20260811 11:57:52.548280 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.143.148:42401/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:0
--local_ip_for_outbound_sockets=127.6.143.130
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:39355
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:45061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:57:52.717446 20380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:57:52.717870 20380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:57:52.717983 20380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:57:52.723922 20380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:57:52.724102 20380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:57:52.724231 20380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:57:52.731753 20380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:45061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.6.143.148:42401/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:39355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:57:52.733346 20380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:57:52.734784 20380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:57:52.747560 20390 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:57:52.748065 20393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:57:52.748152 20389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:57:52.749212 20380 server_base.cc:1034] running on GCE node
I20260811 11:57:52.749676 20380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:57:52.750448 20380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:57:52.751605 20380 hybrid_clock.cc:648] HybridClock initialized: now 1786449472751540 us; error 90 us; skew 500 ppm
Aug 11 11:57:52 dist-test-slave-zkpd krb5kdc[19615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449472, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:57:52.755218 20380 init.cc:377] Logged in from keytab as kudu/127.6.143.130@KRBTEST.COM (short username kudu)
I20260811 11:57:52.756270 20380 webserver.cc:460] Webserver started at http://127.6.143.130:33805/ using document root <none> and password file <none>
I20260811 11:57:52.756978 20380 fs_manager.cc:359] Metadata directory not provided
I20260811 11:57:52.757066 20380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:57:52.757290 20380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 11:57:52 dist-test-slave-zkpd krb5kdc[19615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449472, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for HTTP/127.6.143.148@KRBTEST.COM
I20260811 11:57:52.778498 20380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/data/instance:
uuid: "6bd73ab1bfd8447ca467c8568ed47394"
format_stamp: "Formatted at 2026-08-11 11:57:52 on dist-test-slave-zkpd"
server_key: "7c260a387b039dbc2c6a8bafc47ab272"
server_key_iv: "68ae6eb23a9d27e7f9ba3aac51ef14ec"
server_key_version: "kuduclusterkey@0"
I20260811 11:57:52.779343 20380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance:
uuid: "6bd73ab1bfd8447ca467c8568ed47394"
format_stamp: "Formatted at 2026-08-11 11:57:52 on dist-test-slave-zkpd"
server_key: "7c260a387b039dbc2c6a8bafc47ab272"
server_key_iv: "68ae6eb23a9d27e7f9ba3aac51ef14ec"
server_key_version: "kuduclusterkey@0"
I20260811 11:57:52.784977 20380 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 11:57:52.809640 20400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:52.811403 20380 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 11:57:52.811632 20380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "6bd73ab1bfd8447ca467c8568ed47394"
format_stamp: "Formatted at 2026-08-11 11:57:52 on dist-test-slave-zkpd"
server_key: "7c260a387b039dbc2c6a8bafc47ab272"
server_key_iv: "68ae6eb23a9d27e7f9ba3aac51ef14ec"
server_key_version: "kuduclusterkey@0"
I20260811 11:57:52.811914 20380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:57:52.839794 20380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:57:52.843803 20380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:57:52.844205 20380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:57:52.845142 20380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:57:52.846588 20380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:57:52.846699 20380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:52.846819 20380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:57:52.846897 20380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:52.860097 20380 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:36799
I20260811 11:57:52.861104 20380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
I20260811 11:57:52.862669 20522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:36799 every 8 connection(s)
I20260811 11:57:52.865545 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 20380
I20260811 11:57:52.865669 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance
I20260811 11:57:52.865988 6718 external_mini_cluster.cc:1428] Setting key 560c20125129b7960640a185ee509858
Aug 11 11:57:52 dist-test-slave-zkpd krb5kdc[19615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449472, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:57:52.880339 20376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:52.864495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:60547 (local address 127.6.143.190:39355)
0811 11:57:52.864660 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:52.864665 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:52.866711 (+ 2046us) server_negotiation.cc:407] Connection header received
0811 11:57:52.867964 (+ 1253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:52.867968 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:52.868026 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:52.868080 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:57:52.869777 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:52.870368 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:52.871476 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:52.871873 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:52.872016 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:52.874819 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:57:52.874825 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:57:52.874831 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:57:52.874865 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:57:52.876811 (+ 1946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:52.877507 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:52.877510 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:52.877514 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:52.877609 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:52.877994 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:52.877999 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:52.878001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:52.878184 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:57:52.878306 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:52.880020 (+ 1714us) server_negotiation.cc:299] Negotiation successful
0811 11:57:52.880159 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 11:57:52.881395 20525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:52.864910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39355 (local address 127.6.143.130:60547)
0811 11:57:52.866562 (+ 1652us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:52.866621 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 11:57:52.867689 (+ 1068us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:57:52.868511 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:57:52.868524 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:57:52.868897 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:57:52.869595 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:52.869623 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:52.870496 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:52.870498 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:52.871350 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:52.871358 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:52.872215 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:52.872221 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:52.872476 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:52.873156 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:57:52.873185 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:57:52.874646 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:57:52.876959 (+ 2313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:52.876967 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:52.876983 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:52.877379 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:52.877720 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:52.877726 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:52.877731 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:52.877884 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:52.878296 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:57:52.878302 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:57:52.878642 (+ 340us) client_negotiation.cc:769] Sending connection context
0811 11:57:52.878888 (+ 246us) client_negotiation.cc:240] Negotiation successful
0811 11:57:52.879271 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":150,"threads_started":1}
WARNING: no policy specified for kudu/127.6.143.131@KRBTEST.COM; defaulting to no policy
I20260811 11:57:52.883069 20523 heartbeater.cc:347] Connected to a master server at 127.6.143.190:39355
I20260811 11:57:52.883315 20523 heartbeater.cc:464] Registering TS with master...
I20260811 11:57:52.884050 20523 heartbeater.cc:511] Master 127.6.143.190:39355 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.131@KRBTEST.COM" created.
I20260811 11:57:52.885356 20137 ts_manager.cc:194] Registered new tserver with Master: 6bd73ab1bfd8447ca467c8568ed47394 (127.6.143.130:36799)
I20260811 11:57:52.886159 20137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.130@KRBTEST.COM'} at 127.6.143.130:60547
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
I20260811 11:57:52.913125 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.143.148:42401/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:0
--local_ip_for_outbound_sockets=127.6.143.131
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:39355
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:45061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 11:57:53 dist-test-slave-zkpd krb5kdc[19615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449472, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for HTTP/127.6.143.148@KRBTEST.COM
W20260811 11:57:53.094533 20528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:57:53.094960 20528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:57:53.095079 20528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:57:53.101316 20528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:57:53.101464 20528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:57:53.101605 20528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:57:53.109278 20528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:45061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.6.143.148:42401/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:39355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:57:53.110905 20528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:57:53.112372 20528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:57:53.125353 20536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:57:53.126606 20528 server_base.cc:1034] running on GCE node
W20260811 11:57:53.127852 20538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:57:53.128166 20534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:57:53.128707 20528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:57:53.129499 20528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:57:53.130761 20528 hybrid_clock.cc:648] HybridClock initialized: now 1786449473130693 us; error 79 us; skew 500 ppm
Aug 11 11:57:53 dist-test-slave-zkpd krb5kdc[19615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449473, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:57:53.134549 20528 init.cc:377] Logged in from keytab as kudu/127.6.143.131@KRBTEST.COM (short username kudu)
I20260811 11:57:53.135569 20528 webserver.cc:460] Webserver started at http://127.6.143.131:33481/ using document root <none> and password file <none>
I20260811 11:57:53.136294 20528 fs_manager.cc:359] Metadata directory not provided
I20260811 11:57:53.136373 20528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:57:53.136601 20528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 11:57:53 dist-test-slave-zkpd krb5kdc[19615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449473, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for HTTP/127.6.143.148@KRBTEST.COM
I20260811 11:57:53.163529 20528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/data/instance:
uuid: "c6e794b1469b486280bc99dedfda9fac"
format_stamp: "Formatted at 2026-08-11 11:57:53 on dist-test-slave-zkpd"
server_key: "050bb3329261079107a4f3872f04a7a4"
server_key_iv: "8a5257410c9830ef992a712d7687dc82"
server_key_version: "kuduclusterkey@0"
I20260811 11:57:53.164575 20528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance:
uuid: "c6e794b1469b486280bc99dedfda9fac"
format_stamp: "Formatted at 2026-08-11 11:57:53 on dist-test-slave-zkpd"
server_key: "050bb3329261079107a4f3872f04a7a4"
server_key_iv: "8a5257410c9830ef992a712d7687dc82"
server_key_version: "kuduclusterkey@0"
I20260811 11:57:53.172851 20528 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.004s sys 0.000s
I20260811 11:57:53.212313 20545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:53.214216 20528 fs_manager.cc:725] Time spent opening block manager: real 0.015s user 0.004s sys 0.000s
I20260811 11:57:53.214350 20528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "c6e794b1469b486280bc99dedfda9fac"
format_stamp: "Formatted at 2026-08-11 11:57:53 on dist-test-slave-zkpd"
server_key: "050bb3329261079107a4f3872f04a7a4"
server_key_iv: "8a5257410c9830ef992a712d7687dc82"
server_key_version: "kuduclusterkey@0"
I20260811 11:57:53.214488 20528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:57:53.238052 20528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:57:53.241670 20528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:57:53.241917 20528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:57:53.242636 20528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:57:53.243700 20528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:57:53.243817 20528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:53.243932 20528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:57:53.243999 20528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:53.255417 20528 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:40805
I20260811 11:57:53.256568 20528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:57:53.259172 20670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:40805 every 8 connection(s)
I20260811 11:57:53.262722 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 20528
I20260811 11:57:53.262849 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance
I20260811 11:57:53.263181 6718 external_mini_cluster.cc:1428] Setting key 2f219918b84b2dbb2d8ed9ad052e8d8e
Aug 11 11:57:53 dist-test-slave-zkpd krb5kdc[19615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449473, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:57:53.280419 20376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:53.263102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:56985 (local address 127.6.143.190:39355)
0811 11:57:53.263274 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:53.263279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:53.265832 (+ 2553us) server_negotiation.cc:407] Connection header received
0811 11:57:53.267021 (+ 1189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:53.267024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:53.267078 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:53.267127 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:57:53.268818 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:53.269626 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:53.271285 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:53.271608 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:53.271685 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:53.274715 (+ 3030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:57:53.274719 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:57:53.274722 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:57:53.274758 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:57:53.278017 (+ 3259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:53.278618 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:53.278621 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:53.278623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:53.278708 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:53.279116 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:53.279120 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:53.279121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:53.279354 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:57:53.279440 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:53.280133 (+ 693us) server_negotiation.cc:299] Negotiation successful
0811 11:57:53.280264 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 11:57:53.282236 20675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:53.261939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39355 (local address 127.6.143.131:56985)
0811 11:57:53.265679 (+ 3740us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:53.265731 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 11:57:53.266786 (+ 1055us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:57:53.267564 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:57:53.267574 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:57:53.267986 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:57:53.268635 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:53.268654 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:53.269803 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:53.269806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:53.271145 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:53.271157 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:53.271916 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:53.271921 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:53.272111 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:53.272791 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:57:53.272813 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:57:53.274522 (+ 1709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:57:53.278158 (+ 3636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:53.278164 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:53.278180 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:53.278504 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:53.278826 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:53.278829 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:53.278831 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:53.279010 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:53.279511 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:57:53.279517 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:57:53.279822 (+ 305us) client_negotiation.cc:769] Sending connection context
0811 11:57:53.280861 (+ 1039us) client_negotiation.cc:240] Negotiation successful
0811 11:57:53.281187 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3525,"thread_start_us":146,"threads_started":1}
I20260811 11:57:53.283633 20672 heartbeater.cc:347] Connected to a master server at 127.6.143.190:39355
I20260811 11:57:53.283876 20672 heartbeater.cc:464] Registering TS with master...
I20260811 11:57:53.284564 20672 heartbeater.cc:511] Master 127.6.143.190:39355 requested a full tablet report, sending...
I20260811 11:57:53.285704 20137 ts_manager.cc:194] Registered new tserver with Master: c6e794b1469b486280bc99dedfda9fac (127.6.143.131:40805)
I20260811 11:57:53.286465 20137 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.131@KRBTEST.COM'} at 127.6.143.131:56985
I20260811 11:57:53.290050 6718 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:57:53.291004 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 20227
I20260811 11:57:53.324630 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 20380
I20260811 11:57:53.334568 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 20528
I20260811 11:57:53.344579 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 20101
I20260811 11:57:53.351563 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:39355
--webserver_interface=127.6.143.190
--webserver_port=46475
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:45061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:39355
--ranger_config_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.143.148:42401/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.143.148:42401/kms with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:57:53.467505 20676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:57:53.467763 20676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:57:53.467828 20676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:57:53.471848 20676 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:57:53.471946 20676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:57:53.471992 20676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:57:53.472028 20676 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:57:53.472059 20676 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:57:53.477444 20676 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:45061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.143.148:42401/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:39355
--ranger_config_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:39355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=46475
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:57:53.478533 20676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:57:53.479573 20676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:57:53.489270 20686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:57:53.489336 20681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:57:53.489336 20682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:57:53.489550 20676 server_base.cc:1034] running on GCE node
I20260811 11:57:53.490214 20676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:57:53.491288 20676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:57:53.492475 20676 hybrid_clock.cc:648] HybridClock initialized: now 1786449473492444 us; error 51 us; skew 500 ppm
Aug 11 11:57:53 dist-test-slave-zkpd krb5kdc[19615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449473, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:57:53.495203 20676 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:57:53.496208 20676 webserver.cc:460] Webserver started at http://127.6.143.190:46475/ using document root <none> and password file <none>
I20260811 11:57:53.496744 20676 fs_manager.cc:359] Metadata directory not provided
I20260811 11:57:53.496814 20676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 11:57:53 dist-test-slave-zkpd krb5kdc[19615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449473, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for HTTP/127.6.143.148@KRBTEST.COM
I20260811 11:57:53.576081 20676 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 11:57:53.578384 20694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:53.579809 20676 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 11:57:53.579967 20676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "e25823411ce8459c8fa98e4f8d7b4174"
format_stamp: "Formatted at 2026-08-11 11:57:51 on dist-test-slave-zkpd"
server_key: "8cdf99a4e5e4b3df1d07e812baad2f04"
server_key_iv: "4bd073ca3ee67c4176fdf3bd66c5b506"
server_key_version: "kuduclusterkey@0"
I20260811 11:57:53.580518 20676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:57:53.593217 20676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:57:53.596223 20676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:57:53.596426 20676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:57:53.604660 20676 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:39355
I20260811 11:57:53.604673 20748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:39355 every 8 connection(s)
I20260811 11:57:53.605801 20676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:57:53.609540 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 20676
I20260811 11:57:53.610041 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.143.148:42401/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:38515
--local_ip_for_outbound_sockets=127.6.143.129
--tserver_master_addrs=127.6.143.190:39355
--webserver_port=34615
--webserver_interface=127.6.143.129
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:45061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.143.148:42401/kms with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:57:53.610601 20750 sys_catalog.cc:263] Verifying existing consensus state
I20260811 11:57:53.612902 20750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174: Bootstrap starting.
I20260811 11:57:53.636126 20750 log.cc:824] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174: Log is configured to *not* fsync() on all Append() calls
I20260811 11:57:53.644972 20750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 11:57:53.645423 20750 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174: Bootstrap complete.
I20260811 11:57:53.650977 20750 raft_consensus.cc:348] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e25823411ce8459c8fa98e4f8d7b4174" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39355 } }
I20260811 11:57:53.651485 20750 raft_consensus.cc:729] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e25823411ce8459c8fa98e4f8d7b4174, State: Initialized, Role: FOLLOWER
I20260811 11:57:53.652060 20750 consensus_queue.cc:260] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e25823411ce8459c8fa98e4f8d7b4174" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39355 } }
I20260811 11:57:53.652204 20750 raft_consensus.cc:388] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:57:53.652273 20750 raft_consensus.cc:482] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:57:53.652372 20750 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [term 1 FOLLOWER]: Advancing to term 2
I20260811 11:57:53.654829 20750 raft_consensus.cc:504] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e25823411ce8459c8fa98e4f8d7b4174" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39355 } }
I20260811 11:57:53.655207 20750 leader_election.cc:302] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e25823411ce8459c8fa98e4f8d7b4174; no voters:
I20260811 11:57:53.655586 20750 leader_election.cc:288] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 11:57:53.655709 20754 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 11:57:53.656013 20754 raft_consensus.cc:686] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [term 2 LEADER]: Becoming Leader. State: Replica: e25823411ce8459c8fa98e4f8d7b4174, State: Running, Role: LEADER
I20260811 11:57:53.656445 20754 consensus_queue.cc:237] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e25823411ce8459c8fa98e4f8d7b4174" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39355 } }
I20260811 11:57:53.656919 20750 sys_catalog.cc:564] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:57:53.657853 20755 sys_catalog.cc:455] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "e25823411ce8459c8fa98e4f8d7b4174" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e25823411ce8459c8fa98e4f8d7b4174" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39355 } } }
I20260811 11:57:53.657985 20755 sys_catalog.cc:458] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [sys.catalog]: This master's current role is: LEADER
I20260811 11:57:53.658542 20756 sys_catalog.cc:455] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e25823411ce8459c8fa98e4f8d7b4174. Latest consensus state: current_term: 2 leader_uuid: "e25823411ce8459c8fa98e4f8d7b4174" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e25823411ce8459c8fa98e4f8d7b4174" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39355 } } }
I20260811 11:57:53.658694 20756 sys_catalog.cc:458] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174 [sys.catalog]: This master's current role is: LEADER
I20260811 11:57:53.659022 20764 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:57:53.659255 20750 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 11:57:53.665743 20764 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:57:53.666704 20764 catalog_manager.cc:1259] Loaded cluster ID: 3d18fedeba17477e9bd39d354cfb0b22
I20260811 11:57:53.666762 20764 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:57:53.669500 20764 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:57:53.670264 20764 catalog_manager.cc:5879] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174: Loaded TSK: 0
I20260811 11:57:53.671561 20764 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 11:57:53.748740 20752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:57:53.749075 20752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:57:53.749143 20752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:57:53.755596 20752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:57:53.755672 20752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:57:53.755754 20752 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:57:53.763710 20752 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:45061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.143.148:42401/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:38515
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=34615
--tserver_master_addrs=127.6.143.190:39355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20752
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:57:53.765120 20752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:57:53.766449 20752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:57:53.778354 20796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:57:53.778391 20794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:57:53.778582 20793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:57:53.778687 20752 server_base.cc:1034] running on GCE node
I20260811 11:57:53.779270 20752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:57:53.780049 20752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:57:53.781271 20752 hybrid_clock.cc:648] HybridClock initialized: now 1786449473781243 us; error 32 us; skew 500 ppm
Aug 11 11:57:53 dist-test-slave-zkpd krb5kdc[19615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449473, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:57:53.784785 20752 init.cc:377] Logged in from keytab as kudu/127.6.143.129@KRBTEST.COM (short username kudu)
I20260811 11:57:53.785907 20752 webserver.cc:460] Webserver started at http://127.6.143.129:34615/ using document root <none> and password file <none>
I20260811 11:57:53.786625 20752 fs_manager.cc:359] Metadata directory not provided
I20260811 11:57:53.786737 20752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 11:57:53 dist-test-slave-zkpd krb5kdc[19615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449473, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for HTTP/127.6.143.148@KRBTEST.COM
I20260811 11:57:53.828178 20752 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.004s sys 0.000s
I20260811 11:57:53.831815 20804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:53.833768 20752 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 11:57:53.833894 20752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "24989c5801fc4f4d8b2a2c3c20515386"
format_stamp: "Formatted at 2026-08-11 11:57:52 on dist-test-slave-zkpd"
server_key: "066560adb1dbe89b5c64d257fd4f2205"
server_key_iv: "413cc37ea248184a1dd18996541b5de8"
server_key_version: "kuduclusterkey@0"
I20260811 11:57:53.834573 20752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:57:53.875178 20752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:57:53.879102 20752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:57:53.879356 20752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:57:53.880154 20752 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:57:53.881274 20752 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:57:53.881382 20752 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:53.881520 20752 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:57:53.881605 20752 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:53.893491 20752 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:38515
I20260811 11:57:53.894557 20752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:57:53.897289 20924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:38515 every 8 connection(s)
I20260811 11:57:53.900213 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 20752
I20260811 11:57:53.900728 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.143.148:42401/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:36799
--local_ip_for_outbound_sockets=127.6.143.130
--tserver_master_addrs=127.6.143.190:39355
--webserver_port=33805
--webserver_interface=127.6.143.130
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:45061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.143.148:42401/kms with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 11:57:53 dist-test-slave-zkpd krb5kdc[19615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449473, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:57:53.916138 20928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:53.898139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:33587 (local address 127.6.143.190:39355)
0811 11:57:53.900079 (+ 1940us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:53.900105 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:53.900156 (+ 51us) server_negotiation.cc:407] Connection header received
0811 11:57:53.900463 (+ 307us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:53.900478 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:53.900864 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:53.901197 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:57:53.902862 (+ 1665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:53.904391 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:53.906115 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:53.906847 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:53.907034 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:53.909832 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:57:53.909838 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:57:53.909855 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:57:53.909888 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:57:53.912082 (+ 2194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:53.912743 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:53.912750 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:53.912756 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:53.912892 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:53.913343 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:53.913347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:53.913349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:53.913856 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:57:53.913965 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:53.914721 (+ 756us) server_negotiation.cc:299] Negotiation successful
0811 11:57:53.915035 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":137,"threads_started":1}
I20260811 11:57:53.918043 20927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:53.898184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39355 (local address 127.6.143.129:33587)
0811 11:57:53.898662 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:53.898717 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 11:57:53.899599 (+ 882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:57:53.901681 (+ 2082us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:57:53.901691 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:57:53.902019 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:57:53.902643 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:53.902665 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:53.904547 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:53.904551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:53.905964 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:53.905976 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:53.907031 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:53.907036 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:53.907266 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:53.907992 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:57:53.908014 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:57:53.909599 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:57:53.912242 (+ 2643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:53.912248 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:53.912264 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:53.912595 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:53.913012 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:53.913015 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:53.913017 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:53.913220 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:53.914049 (+ 829us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:57:53.914055 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:57:53.914358 (+ 303us) client_negotiation.cc:769] Sending connection context
0811 11:57:53.916850 (+ 2492us) client_negotiation.cc:240] Negotiation successful
0811 11:57:53.917169 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":147,"threads_started":1}
I20260811 11:57:53.921154 20925 heartbeater.cc:347] Connected to a master server at 127.6.143.190:39355
I20260811 11:57:53.921427 20925 heartbeater.cc:464] Registering TS with master...
I20260811 11:57:53.922089 20925 heartbeater.cc:511] Master 127.6.143.190:39355 requested a full tablet report, sending...
W20260811 11:57:53.924624 20925 heartbeater.cc:644] Failed to heartbeat to 127.6.143.190:39355 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 11:57:53.924755 20925 heartbeater.cc:464] Registering TS with master...
I20260811 11:57:53.924916 20925 heartbeater.cc:511] Master 127.6.143.190:39355 requested a full tablet report, sending...
I20260811 11:57:53.925788 20925 heartbeater.cc:464] Registering TS with master...
I20260811 11:57:53.925944 20925 heartbeater.cc:511] Master 127.6.143.190:39355 requested a full tablet report, sending...
W20260811 11:57:53.926687 20925 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 11:57:54.040899 20929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:57:54.041172 20929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:57:54.041283 20929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:57:54.045437 20929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:57:54.045516 20929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:57:54.045606 20929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:57:54.050824 20929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:45061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.143.148:42401/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:36799
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=33805
--tserver_master_addrs=127.6.143.190:39355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:57:54.051937 20929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:57:54.053025 20929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 11:57:54.063866 20929 server_base.cc:1034] running on GCE node
W20260811 11:57:54.063818 20934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:57:54.063830 20937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:57:54.064126 20935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:57:54.064584 20929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:57:54.065254 20929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:57:54.066435 20929 hybrid_clock.cc:648] HybridClock initialized: now 1786449474066408 us; error 47 us; skew 500 ppm
Aug 11 11:57:54 dist-test-slave-zkpd krb5kdc[19615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449474, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:57:54.069464 20929 init.cc:377] Logged in from keytab as kudu/127.6.143.130@KRBTEST.COM (short username kudu)
I20260811 11:57:54.070271 20929 webserver.cc:460] Webserver started at http://127.6.143.130:33805/ using document root <none> and password file <none>
I20260811 11:57:54.070844 20929 fs_manager.cc:359] Metadata directory not provided
I20260811 11:57:54.070956 20929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 11:57:54 dist-test-slave-zkpd krb5kdc[19615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449474, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for HTTP/127.6.143.148@KRBTEST.COM
I20260811 11:57:54.106302 20929 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 11:57:54.111856 20946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:54.113487 20929 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.002s sys 0.000s
I20260811 11:57:54.113607 20929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "6bd73ab1bfd8447ca467c8568ed47394"
format_stamp: "Formatted at 2026-08-11 11:57:52 on dist-test-slave-zkpd"
server_key: "7c260a387b039dbc2c6a8bafc47ab272"
server_key_iv: "68ae6eb23a9d27e7f9ba3aac51ef14ec"
server_key_version: "kuduclusterkey@0"
I20260811 11:57:54.114349 20929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:57:54.129377 20929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:57:54.133436 20929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:57:54.133656 20929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:57:54.134296 20929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:57:54.135157 20929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:57:54.135231 20929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:54.135327 20929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:57:54.135365 20929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:54.146922 20929 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:36799
I20260811 11:57:54.148308 20929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
I20260811 11:57:54.149147 21081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:36799 every 8 connection(s)
Aug 11 11:57:54 dist-test-slave-zkpd krb5kdc[19615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449474, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:57:54.158337 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 20929
I20260811 11:57:54.158876 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.143.148:42401/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:40805
--local_ip_for_outbound_sockets=127.6.143.131
--tserver_master_addrs=127.6.143.190:39355
--webserver_port=33481
--webserver_interface=127.6.143.131
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:45061
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.143.148:42401/kms with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:57:54.163342 20928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:54.150351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:44461 (local address 127.6.143.190:39355)
0811 11:57:54.150516 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:54.150521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:54.151450 (+ 929us) server_negotiation.cc:407] Connection header received
0811 11:57:54.152610 (+ 1160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:54.152614 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:54.152673 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:54.152784 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:57:54.153888 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:54.154421 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:54.155564 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:54.155795 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:54.155904 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:54.158772 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:57:54.158779 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:57:54.158785 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:57:54.158817 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:57:54.160890 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:54.161568 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:54.161575 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:54.161577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:54.161673 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:54.162108 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:54.162112 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:54.162113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:54.162335 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:57:54.162409 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:54.163024 (+ 615us) server_negotiation.cc:299] Negotiation successful
0811 11:57:54.163160 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 11:57:54.164491 21084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:54.150819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39355 (local address 127.6.143.130:44461)
0811 11:57:54.151303 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:54.151354 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 11:57:54.152346 (+ 992us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:57:54.152970 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:57:54.152977 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:57:54.153241 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:57:54.153724 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:54.153737 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:54.154549 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:54.154552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:54.155421 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:54.155431 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:54.155969 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:54.155973 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:54.156143 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:54.156743 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:57:54.156765 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:57:54.158557 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:57:54.161028 (+ 2471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:54.161037 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:54.161056 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:54.161430 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:54.161807 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:54.161810 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:54.161812 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:54.161989 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:54.162496 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:57:54.162502 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:57:54.162784 (+ 282us) client_negotiation.cc:769] Sending connection context
0811 11:57:54.162979 (+ 195us) client_negotiation.cc:240] Negotiation successful
0811 11:57:54.163284 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":154,"threads_started":1}
I20260811 11:57:54.166198 21082 heartbeater.cc:347] Connected to a master server at 127.6.143.190:39355
I20260811 11:57:54.166500 21082 heartbeater.cc:464] Registering TS with master...
I20260811 11:57:54.167177 21082 heartbeater.cc:511] Master 127.6.143.190:39355 requested a full tablet report, sending...
W20260811 11:57:54.168905 21082 heartbeater.cc:644] Failed to heartbeat to 127.6.143.190:39355 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 11:57:54.169080 21082 heartbeater.cc:464] Registering TS with master...
I20260811 11:57:54.169291 21082 heartbeater.cc:511] Master 127.6.143.190:39355 requested a full tablet report, sending...
I20260811 11:57:54.170168 21082 heartbeater.cc:464] Registering TS with master...
I20260811 11:57:54.170384 21082 heartbeater.cc:511] Master 127.6.143.190:39355 requested a full tablet report, sending...
W20260811 11:57:54.171213 21082 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 11:57:54.301532 21087 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:57:54.301973 21087 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:57:54.302162 21087 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:57:54.308342 21087 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:57:54.308411 21087 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:57:54.308497 21087 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:57:54.316239 21087 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:45061
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.143.148:42401/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:40805
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=33481
--tserver_master_addrs=127.6.143.190:39355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21087
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:57:54.317690 21087 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:57:54.318995 21087 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:57:54.331485 21096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:57:54.331476 21094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:57:54.331590 21092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:57:54.331852 21087 server_base.cc:1034] running on GCE node
I20260811 11:57:54.332275 21087 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:57:54.332930 21087 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:57:54.334142 21087 hybrid_clock.cc:648] HybridClock initialized: now 1786449474334096 us; error 42 us; skew 500 ppm
Aug 11 11:57:54 dist-test-slave-zkpd krb5kdc[19615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449474, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:57:54.336985 21087 init.cc:377] Logged in from keytab as kudu/127.6.143.131@KRBTEST.COM (short username kudu)
I20260811 11:57:54.337992 21087 webserver.cc:460] Webserver started at http://127.6.143.131:33481/ using document root <none> and password file <none>
I20260811 11:57:54.338555 21087 fs_manager.cc:359] Metadata directory not provided
I20260811 11:57:54.338658 21087 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 11:57:54 dist-test-slave-zkpd krb5kdc[19615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449474, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for HTTP/127.6.143.148@KRBTEST.COM
I20260811 11:57:54.359339 21087 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 11:57:54.362958 21105 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:54.364715 21087 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 11:57:54.364943 21087 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "c6e794b1469b486280bc99dedfda9fac"
format_stamp: "Formatted at 2026-08-11 11:57:53 on dist-test-slave-zkpd"
server_key: "050bb3329261079107a4f3872f04a7a4"
server_key_iv: "8a5257410c9830ef992a712d7687dc82"
server_key_version: "kuduclusterkey@0"
I20260811 11:57:54.365731 21087 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:57:54.390982 21087 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:57:54.394950 21087 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:57:54.395205 21087 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:57:54.395974 21087 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:57:54.397047 21087 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:57:54.397153 21087 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:54.397259 21087 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:57:54.397330 21087 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:54.417831 21087 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:40805
I20260811 11:57:54.417974 21231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:40805 every 8 connection(s)
I20260811 11:57:54.419046 21087 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:57:54.428761 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 21087
Aug 11 11:57:54 dist-test-slave-zkpd krb5kdc[19615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449474, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:57:54.435485 20928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:54.421207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:48921 (local address 127.6.143.190:39355)
0811 11:57:54.421340 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:54.421343 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:54.422483 (+ 1140us) server_negotiation.cc:407] Connection header received
0811 11:57:54.423552 (+ 1069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:54.423556 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:54.423617 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:54.423718 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:57:54.424952 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:54.425503 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:54.426719 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:54.427093 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:54.427210 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:54.430998 (+ 3788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:57:54.431001 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:57:54.431003 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:57:54.431028 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:57:54.433058 (+ 2030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:54.433648 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:54.433652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:54.433653 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:54.433794 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:54.434233 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:54.434237 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:54.434239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:54.434461 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:57:54.434540 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:54.435250 (+ 710us) server_negotiation.cc:299] Negotiation successful
0811 11:57:54.435355 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 11:57:54.437779 21236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:54.421729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39355 (local address 127.6.143.131:48921)
0811 11:57:54.422301 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:54.422357 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 11:57:54.423299 (+ 942us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:57:54.423875 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:57:54.423882 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:57:54.424206 (+ 324us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:57:54.424797 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:54.424813 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:54.425649 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:54.425652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:54.426582 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:54.426594 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:54.427388 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:54.427392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:54.427593 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:54.428279 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:57:54.428300 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:57:54.430694 (+ 2394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:57:54.433160 (+ 2466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:54.433166 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:54.433181 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:54.433522 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:54.433924 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:54.433927 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:54.433929 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:54.434107 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:54.434628 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:57:54.434635 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:57:54.434962 (+ 327us) client_negotiation.cc:769] Sending connection context
0811 11:57:54.435189 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 11:57:54.435525 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":135,"threads_started":1}
I20260811 11:57:54.439004 21232 heartbeater.cc:347] Connected to a master server at 127.6.143.190:39355
I20260811 11:57:54.439296 21232 heartbeater.cc:464] Registering TS with master...
I20260811 11:57:54.439881 21232 heartbeater.cc:511] Master 127.6.143.190:39355 requested a full tablet report, sending...
I20260811 11:57:54.441408 20711 ts_manager.cc:194] Registered new tserver with Master: c6e794b1469b486280bc99dedfda9fac (127.6.143.131:40805)
I20260811 11:57:54.443046 21238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:54.429483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55832 (local address 127.6.143.190:39355)
0811 11:57:54.430487 (+ 1004us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:54.430491 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:54.430743 (+ 252us) server_negotiation.cc:407] Connection header received
0811 11:57:54.430948 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:54.430952 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:54.431006 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:54.431100 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:57:54.432140 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:54.432968 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:54.435890 (+ 2922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:54.436232 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:54.436348 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:54.438440 (+ 2092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:57:54.438443 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:57:54.438446 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:57:54.438472 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:57:54.440631 (+ 2159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:54.441144 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:54.441147 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:54.441149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:54.441236 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:54.441602 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:54.441606 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:54.441608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:54.441812 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:57:54.442004 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:54.442752 (+ 748us) server_negotiation.cc:299] Negotiation successful
0811 11:57:54.442874 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":922,"thread_start_us":89,"threads_started":1}
I20260811 11:57:54.443041 20711 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.131@KRBTEST.COM'} at 127.6.143.131:48921
Aug 11 11:57:54 dist-test-slave-zkpd krb5kdc[19615](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786449474, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:57:54.926892 20925 heartbeater.cc:464] Registering TS with master...
I20260811 11:57:54.927223 20925 heartbeater.cc:511] Master 127.6.143.190:39355 requested a full tablet report, sending...
I20260811 11:57:54.928058 20710 ts_manager.cc:194] Registered new tserver with Master: 24989c5801fc4f4d8b2a2c3c20515386 (127.6.143.129:38515)
I20260811 11:57:54.928856 20710 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.129@KRBTEST.COM'} at 127.6.143.129:33587
Aug 11 11:57:55 dist-test-slave-zkpd krb5kdc[19615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449474, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for HTTP/127.6.143.148@KRBTEST.COM
I20260811 11:57:55.171465 21082 heartbeater.cc:464] Registering TS with master...
I20260811 11:57:55.171756 21082 heartbeater.cc:511] Master 127.6.143.190:39355 requested a full tablet report, sending...
I20260811 11:57:55.172880 20710 ts_manager.cc:194] Registered new tserver with Master: 6bd73ab1bfd8447ca467c8568ed47394 (127.6.143.130:36799)
I20260811 11:57:55.173648 20710 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.130@KRBTEST.COM'} at 127.6.143.130:44461
I20260811 11:57:55.179538 6718 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:57:55.206621 21256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:55.188308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55846 (local address 127.6.143.190:39355)
0811 11:57:55.188609 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:55.188615 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:55.192292 (+ 3677us) server_negotiation.cc:407] Connection header received
0811 11:57:55.192443 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:55.192447 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:55.192515 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:55.192627 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:57:55.193737 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:55.194565 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:55.195783 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:55.196186 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:55.196291 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:55.201490 (+ 5199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:57:55.201495 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:57:55.201498 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:57:55.201563 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:57:55.204331 (+ 2768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:55.204893 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:55.204897 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:55.204899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:55.204989 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:55.205287 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:55.205290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:55.205292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:55.205567 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:57:55.205691 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:55.206315 (+ 624us) server_negotiation.cc:299] Negotiation successful
0811 11:57:55.206442 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":106,"threads_started":1}
I20260811 11:57:55.211314 20710 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55846:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:57:55.216663 20710 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:57:55.237848 21262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:55.229024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:38515 (local address 127.0.0.1:35874)
0811 11:57:55.231323 (+ 2299us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:55.231332 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0811 11:57:55.231439 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:57:55.232155 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:57:55.232161 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:57:55.232199 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:57:55.232346 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:55.232353 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:55.234182 (+ 1829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:55.234185 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:55.235928 (+ 1743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:55.235941 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:55.237276 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:55.237280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:55.237387 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:55.237427 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 11:57:55.237540 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 11:57:55.237683 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2253,"mutex_wait_us":138,"thread_start_us":88,"threads_started":1}
I20260811 11:57:55.238329 21265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:55.229591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35874 (local address 127.6.143.129:38515)
0811 11:57:55.230802 (+ 1211us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:55.230808 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:55.231433 (+ 625us) server_negotiation.cc:407] Connection header received
0811 11:57:55.231560 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:55.231566 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:55.231801 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:55.231982 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:57:55.232487 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:55.234035 (+ 1548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:55.236085 (+ 2050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:55.236941 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:55.237017 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:55.237161 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:55.238064 (+ 903us) server_negotiation.cc:299] Negotiation successful
0811 11:57:55.238178 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1109,"thread_start_us":110,"threads_started":1}
I20260811 11:57:55.240564 21264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:55.229359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:40805 (local address 127.0.0.1:32906)
0811 11:57:55.231621 (+ 2262us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:55.231633 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:57:55.231723 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:57:55.232413 (+ 690us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:57:55.232416 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:57:55.232436 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:57:55.232519 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:55.232527 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:55.234287 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:55.234290 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:55.236641 (+ 2351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:55.236652 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:55.239331 (+ 2679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:55.239335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:55.239422 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:55.239447 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 11:57:55.239523 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 11:57:55.239595 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2205,"mutex_wait_us":900,"thread_start_us":76,"threads_started":1}
I20260811 11:57:55.241309 21267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:55.229978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32906 (local address 127.6.143.131:40805)
0811 11:57:55.230314 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:55.230322 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:55.231820 (+ 1498us) server_negotiation.cc:407] Connection header received
0811 11:57:55.231927 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:55.231933 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:55.232135 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:55.232277 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:57:55.232647 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:55.234169 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:55.236827 (+ 2658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:55.237621 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:55.237735 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:55.237874 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:55.241065 (+ 3191us) server_negotiation.cc:299] Negotiation successful
0811 11:57:55.241174 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":142,"threads_started":1}
I20260811 11:57:55.241953 20835 tablet_service.cc:1467] Processing CreateTablet for tablet 94022030759c45eaac0ca14440bb8743 (DEFAULT_TABLE table=test-table [id=d4015880e922423890a0b08c80527ee2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:57:55.243130 20835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94022030759c45eaac0ca14440bb8743. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:57:55.244517 21152 tablet_service.cc:1467] Processing CreateTablet for tablet 94022030759c45eaac0ca14440bb8743 (DEFAULT_TABLE table=test-table [id=d4015880e922423890a0b08c80527ee2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:57:55.245568 21152 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94022030759c45eaac0ca14440bb8743. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:57:55.252285 21268 tablet_bootstrap.cc:492] T 94022030759c45eaac0ca14440bb8743 P 24989c5801fc4f4d8b2a2c3c20515386: Bootstrap starting.
I20260811 11:57:55.253796 21266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:55.229761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35726 (local address 127.6.143.130:36799)
0811 11:57:55.236099 (+ 6338us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:55.236106 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:55.236133 (+ 27us) server_negotiation.cc:407] Connection header received
0811 11:57:55.236211 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:55.236216 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:55.236409 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:55.248070 (+ 11661us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:57:55.248385 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:55.249529 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:55.252061 (+ 2532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:55.252870 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:55.252957 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:55.253098 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:55.253522 (+ 424us) server_negotiation.cc:299] Negotiation successful
0811 11:57:55.253627 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6220,"thread_start_us":123,"threads_started":1}
I20260811 11:57:55.254379 21263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:55.228628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:36799 (local address 127.0.0.1:35726)
0811 11:57:55.231017 (+ 2389us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:55.231053 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0811 11:57:55.231157 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:57:55.248041 (+ 16884us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:57:55.248045 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:57:55.248072 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:57:55.248211 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:55.248220 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:55.249699 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:55.249702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:55.251851 (+ 2149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:55.251862 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:55.253247 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:55.253250 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:55.253355 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:55.253380 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 11:57:55.254155 (+ 775us) client_negotiation.cc:240] Negotiation successful
0811 11:57:55.254243 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2244,"thread_start_us":93,"threads_started":1}
I20260811 11:57:55.254794 21268 tablet_bootstrap.cc:654] T 94022030759c45eaac0ca14440bb8743 P 24989c5801fc4f4d8b2a2c3c20515386: Neither blocks nor log segments found. Creating new log.
I20260811 11:57:55.255705 21268 log.cc:824] T 94022030759c45eaac0ca14440bb8743 P 24989c5801fc4f4d8b2a2c3c20515386: Log is configured to *not* fsync() on all Append() calls
I20260811 11:57:55.255839 21269 tablet_bootstrap.cc:492] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac: Bootstrap starting.
I20260811 11:57:55.257797 20995 tablet_service.cc:1467] Processing CreateTablet for tablet 94022030759c45eaac0ca14440bb8743 (DEFAULT_TABLE table=test-table [id=d4015880e922423890a0b08c80527ee2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:57:55.258528 21269 tablet_bootstrap.cc:654] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac: Neither blocks nor log segments found. Creating new log.
I20260811 11:57:55.258975 20995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94022030759c45eaac0ca14440bb8743. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:57:55.259651 21269 log.cc:824] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac: Log is configured to *not* fsync() on all Append() calls
I20260811 11:57:55.261611 21269 tablet_bootstrap.cc:492] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac: No bootstrap required, opened a new log
I20260811 11:57:55.261838 21269 ts_tablet_manager.cc:1397] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 11:57:55.265018 21269 raft_consensus.cc:348] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bd73ab1bfd8447ca467c8568ed47394" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36799 } } peers { permanent_uuid: "c6e794b1469b486280bc99dedfda9fac" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 40805 } } peers { permanent_uuid: "24989c5801fc4f4d8b2a2c3c20515386" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 38515 } }
I20260811 11:57:55.265403 21269 raft_consensus.cc:374] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:57:55.265522 21269 raft_consensus.cc:729] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c6e794b1469b486280bc99dedfda9fac, State: Initialized, Role: FOLLOWER
I20260811 11:57:55.265820 21268 tablet_bootstrap.cc:492] T 94022030759c45eaac0ca14440bb8743 P 24989c5801fc4f4d8b2a2c3c20515386: No bootstrap required, opened a new log
I20260811 11:57:55.266026 21268 ts_tablet_manager.cc:1397] T 94022030759c45eaac0ca14440bb8743 P 24989c5801fc4f4d8b2a2c3c20515386: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.000s
I20260811 11:57:55.266685 21269 consensus_queue.cc:260] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bd73ab1bfd8447ca467c8568ed47394" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36799 } } peers { permanent_uuid: "c6e794b1469b486280bc99dedfda9fac" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 40805 } } peers { permanent_uuid: "24989c5801fc4f4d8b2a2c3c20515386" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 38515 } }
I20260811 11:57:55.267839 21269 ts_tablet_manager.cc:1428] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 11:57:55.268265 21232 heartbeater.cc:503] Master 127.6.143.190:39355 was elected leader, sending a full tablet report...
I20260811 11:57:55.270170 21268 raft_consensus.cc:348] T 94022030759c45eaac0ca14440bb8743 P 24989c5801fc4f4d8b2a2c3c20515386 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bd73ab1bfd8447ca467c8568ed47394" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36799 } } peers { permanent_uuid: "c6e794b1469b486280bc99dedfda9fac" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 40805 } } peers { permanent_uuid: "24989c5801fc4f4d8b2a2c3c20515386" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 38515 } }
I20260811 11:57:55.270443 21268 raft_consensus.cc:374] T 94022030759c45eaac0ca14440bb8743 P 24989c5801fc4f4d8b2a2c3c20515386 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:57:55.270502 21268 raft_consensus.cc:729] T 94022030759c45eaac0ca14440bb8743 P 24989c5801fc4f4d8b2a2c3c20515386 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24989c5801fc4f4d8b2a2c3c20515386, State: Initialized, Role: FOLLOWER
I20260811 11:57:55.270964 21268 consensus_queue.cc:260] T 94022030759c45eaac0ca14440bb8743 P 24989c5801fc4f4d8b2a2c3c20515386 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bd73ab1bfd8447ca467c8568ed47394" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36799 } } peers { permanent_uuid: "c6e794b1469b486280bc99dedfda9fac" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 40805 } } peers { permanent_uuid: "24989c5801fc4f4d8b2a2c3c20515386" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 38515 } }
I20260811 11:57:55.272161 21268 ts_tablet_manager.cc:1428] T 94022030759c45eaac0ca14440bb8743 P 24989c5801fc4f4d8b2a2c3c20515386: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 11:57:55.272533 20925 heartbeater.cc:503] Master 127.6.143.190:39355 was elected leader, sending a full tablet report...
I20260811 11:57:55.274679 21272 tablet_bootstrap.cc:492] T 94022030759c45eaac0ca14440bb8743 P 6bd73ab1bfd8447ca467c8568ed47394: Bootstrap starting.
I20260811 11:57:55.277139 21272 tablet_bootstrap.cc:654] T 94022030759c45eaac0ca14440bb8743 P 6bd73ab1bfd8447ca467c8568ed47394: Neither blocks nor log segments found. Creating new log.
I20260811 11:57:55.277979 21272 log.cc:824] T 94022030759c45eaac0ca14440bb8743 P 6bd73ab1bfd8447ca467c8568ed47394: Log is configured to *not* fsync() on all Append() calls
I20260811 11:57:55.279980 21272 tablet_bootstrap.cc:492] T 94022030759c45eaac0ca14440bb8743 P 6bd73ab1bfd8447ca467c8568ed47394: No bootstrap required, opened a new log
I20260811 11:57:55.280184 21272 ts_tablet_manager.cc:1397] T 94022030759c45eaac0ca14440bb8743 P 6bd73ab1bfd8447ca467c8568ed47394: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 11:57:55.283239 21272 raft_consensus.cc:348] T 94022030759c45eaac0ca14440bb8743 P 6bd73ab1bfd8447ca467c8568ed47394 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bd73ab1bfd8447ca467c8568ed47394" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36799 } } peers { permanent_uuid: "c6e794b1469b486280bc99dedfda9fac" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 40805 } } peers { permanent_uuid: "24989c5801fc4f4d8b2a2c3c20515386" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 38515 } }
I20260811 11:57:55.283522 21272 raft_consensus.cc:374] T 94022030759c45eaac0ca14440bb8743 P 6bd73ab1bfd8447ca467c8568ed47394 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:57:55.283587 21272 raft_consensus.cc:729] T 94022030759c45eaac0ca14440bb8743 P 6bd73ab1bfd8447ca467c8568ed47394 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bd73ab1bfd8447ca467c8568ed47394, State: Initialized, Role: FOLLOWER
I20260811 11:57:55.284078 21272 consensus_queue.cc:260] T 94022030759c45eaac0ca14440bb8743 P 6bd73ab1bfd8447ca467c8568ed47394 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bd73ab1bfd8447ca467c8568ed47394" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36799 } } peers { permanent_uuid: "c6e794b1469b486280bc99dedfda9fac" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 40805 } } peers { permanent_uuid: "24989c5801fc4f4d8b2a2c3c20515386" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 38515 } }
I20260811 11:57:55.285382 21272 ts_tablet_manager.cc:1428] T 94022030759c45eaac0ca14440bb8743 P 6bd73ab1bfd8447ca467c8568ed47394: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 11:57:55.285737 21082 heartbeater.cc:503] Master 127.6.143.190:39355 was elected leader, sending a full tablet report...
I20260811 11:57:55.373692 21274 raft_consensus.cc:482] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:57:55.373961 21274 raft_consensus.cc:504] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bd73ab1bfd8447ca467c8568ed47394" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36799 } } peers { permanent_uuid: "c6e794b1469b486280bc99dedfda9fac" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 40805 } } peers { permanent_uuid: "24989c5801fc4f4d8b2a2c3c20515386" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 38515 } }
I20260811 11:57:55.375492 21274 leader_election.cc:288] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6bd73ab1bfd8447ca467c8568ed47394 (127.6.143.130:36799), 24989c5801fc4f4d8b2a2c3c20515386 (127.6.143.129:38515)
I20260811 11:57:55.381790 21265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:55.377058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:42755 (local address 127.6.143.129:38515)
0811 11:57:55.377285 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:55.377292 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:55.377310 (+ 18us) server_negotiation.cc:407] Connection header received
0811 11:57:55.377486 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:55.377492 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:55.377547 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:55.377599 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:57:55.378057 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:55.378787 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:55.380442 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:55.381051 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:55.381142 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:55.381208 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:55.381584 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 11:57:55.381660 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143}
I20260811 11:57:55.382228 21280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:55.376737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:38515 (local address 127.6.143.131:42755)
0811 11:57:55.377219 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:55.377232 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:57:55.377354 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:57:55.377758 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:57:55.377760 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:57:55.377779 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:57:55.377919 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:55.377926 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:55.378920 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:55.378923 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:55.380307 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:55.380316 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:55.381352 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:55.381355 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:55.381449 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:55.381473 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 11:57:55.381986 (+ 513us) client_negotiation.cc:240] Negotiation successful
0811 11:57:55.382071 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"mutex_wait_us":17,"thread_start_us":106,"threads_started":1}
I20260811 11:57:55.382912 21266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:55.376108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:41705 (local address 127.6.143.130:36799)
0811 11:57:55.376240 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:55.376245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:55.378382 (+ 2137us) server_negotiation.cc:407] Connection header received
0811 11:57:55.378560 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:55.378564 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:55.378610 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:55.378652 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:57:55.379119 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:55.379871 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:55.381289 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:55.381858 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:55.381957 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:55.382008 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:55.382721 (+ 713us) server_negotiation.cc:299] Negotiation successful
0811 11:57:55.382803 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 11:57:55.382892 20875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94022030759c45eaac0ca14440bb8743" candidate_uuid: "c6e794b1469b486280bc99dedfda9fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24989c5801fc4f4d8b2a2c3c20515386" is_pre_election: true
I20260811 11:57:55.383229 20875 raft_consensus.cc:2454] T 94022030759c45eaac0ca14440bb8743 P 24989c5801fc4f4d8b2a2c3c20515386 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6e794b1469b486280bc99dedfda9fac in term 0.
I20260811 11:57:55.383570 21278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:55.376299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:36799 (local address 127.6.143.131:41705)
0811 11:57:55.376962 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:55.378305 (+ 1343us) client_negotiation.cc:174] Beginning negotiation
0811 11:57:55.378450 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:57:55.378779 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:57:55.378782 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:57:55.378797 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:57:55.378986 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:55.378993 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:55.380052 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:55.380055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:55.381145 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:55.381156 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:55.382475 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:55.382480 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:55.382572 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:55.382598 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 11:57:55.383254 (+ 656us) client_negotiation.cc:240] Negotiation successful
0811 11:57:55.383403 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"mutex_wait_us":40,"thread_start_us":132,"threads_started":1}
I20260811 11:57:55.383765 21111 leader_election.cc:302] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24989c5801fc4f4d8b2a2c3c20515386, c6e794b1469b486280bc99dedfda9fac; no voters:
I20260811 11:57:55.384145 21274 raft_consensus.cc:2781] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:57:55.384238 21274 raft_consensus.cc:482] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:57:55.384300 21274 raft_consensus.cc:3037] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:57:55.384629 21027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94022030759c45eaac0ca14440bb8743" candidate_uuid: "c6e794b1469b486280bc99dedfda9fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bd73ab1bfd8447ca467c8568ed47394" is_pre_election: true
I20260811 11:57:55.384944 21027 raft_consensus.cc:2454] T 94022030759c45eaac0ca14440bb8743 P 6bd73ab1bfd8447ca467c8568ed47394 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c6e794b1469b486280bc99dedfda9fac in term 0.
I20260811 11:57:55.385686 21274 raft_consensus.cc:504] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bd73ab1bfd8447ca467c8568ed47394" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36799 } } peers { permanent_uuid: "c6e794b1469b486280bc99dedfda9fac" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 40805 } } peers { permanent_uuid: "24989c5801fc4f4d8b2a2c3c20515386" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 38515 } }
I20260811 11:57:55.386065 21274 leader_election.cc:288] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac [CANDIDATE]: Term 1 election: Requested vote from peers 6bd73ab1bfd8447ca467c8568ed47394 (127.6.143.130:36799), 24989c5801fc4f4d8b2a2c3c20515386 (127.6.143.129:38515)
I20260811 11:57:55.386447 20875 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94022030759c45eaac0ca14440bb8743" candidate_uuid: "c6e794b1469b486280bc99dedfda9fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24989c5801fc4f4d8b2a2c3c20515386"
I20260811 11:57:55.386483 21027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94022030759c45eaac0ca14440bb8743" candidate_uuid: "c6e794b1469b486280bc99dedfda9fac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6bd73ab1bfd8447ca467c8568ed47394"
I20260811 11:57:55.386559 20875 raft_consensus.cc:3037] T 94022030759c45eaac0ca14440bb8743 P 24989c5801fc4f4d8b2a2c3c20515386 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:57:55.386602 21027 raft_consensus.cc:3037] T 94022030759c45eaac0ca14440bb8743 P 6bd73ab1bfd8447ca467c8568ed47394 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:57:55.387784 20875 raft_consensus.cc:2454] T 94022030759c45eaac0ca14440bb8743 P 24989c5801fc4f4d8b2a2c3c20515386 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6e794b1469b486280bc99dedfda9fac in term 1.
I20260811 11:57:55.388062 21027 raft_consensus.cc:2454] T 94022030759c45eaac0ca14440bb8743 P 6bd73ab1bfd8447ca467c8568ed47394 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c6e794b1469b486280bc99dedfda9fac in term 1.
I20260811 11:57:55.388080 21111 leader_election.cc:302] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24989c5801fc4f4d8b2a2c3c20515386, c6e794b1469b486280bc99dedfda9fac; no voters:
I20260811 11:57:55.388229 21274 raft_consensus.cc:2781] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:57:55.388439 21274 raft_consensus.cc:686] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac [term 1 LEADER]: Becoming Leader. State: Replica: c6e794b1469b486280bc99dedfda9fac, State: Running, Role: LEADER
I20260811 11:57:55.388764 21274 consensus_queue.cc:237] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bd73ab1bfd8447ca467c8568ed47394" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36799 } } peers { permanent_uuid: "c6e794b1469b486280bc99dedfda9fac" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 40805 } } peers { permanent_uuid: "24989c5801fc4f4d8b2a2c3c20515386" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 38515 } }
I20260811 11:57:55.392093 20711 catalog_manager.cc:5495] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac reported cstate change: term changed from 0 to 1, leader changed from <none> to c6e794b1469b486280bc99dedfda9fac (127.6.143.131). New cstate: current_term: 1 leader_uuid: "c6e794b1469b486280bc99dedfda9fac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6bd73ab1bfd8447ca467c8568ed47394" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36799 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c6e794b1469b486280bc99dedfda9fac" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 40805 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "24989c5801fc4f4d8b2a2c3c20515386" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 38515 } health_report { overall_health: UNKNOWN } } }
W20260811 11:57:55.397429 20926 tablet.cc:2369] T 94022030759c45eaac0ca14440bb8743 P 24989c5801fc4f4d8b2a2c3c20515386: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 11:57:55.402295 21083 tablet.cc:2369] T 94022030759c45eaac0ca14440bb8743 P 6bd73ab1bfd8447ca467c8568ed47394: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 11:57:55.420603 21233 tablet.cc:2369] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:57:55.436702 21267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:55.431568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32918 (local address 127.6.143.131:40805)
0811 11:57:55.431708 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:55.431713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:55.431727 (+ 14us) server_negotiation.cc:407] Connection header received
0811 11:57:55.431802 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:55.431805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:55.431867 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:55.431998 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:57:55.432415 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:55.433197 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:55.434469 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:55.434863 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:55.434978 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:55.435435 (+ 457us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:57:55.435991 (+ 556us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:57:55.436100 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:55.436518 (+ 418us) server_negotiation.cc:299] Negotiation successful
0811 11:57:55.436585 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 11:57:55.445413 20875 raft_consensus.cc:1261] T 94022030759c45eaac0ca14440bb8743 P 24989c5801fc4f4d8b2a2c3c20515386 [term 1 FOLLOWER]: Refusing update from remote peer c6e794b1469b486280bc99dedfda9fac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:57:55.445413 21027 raft_consensus.cc:1261] T 94022030759c45eaac0ca14440bb8743 P 6bd73ab1bfd8447ca467c8568ed47394 [term 1 FOLLOWER]: Refusing update from remote peer c6e794b1469b486280bc99dedfda9fac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
Aug 11 11:57:55 dist-test-slave-zkpd krb5kdc[19615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449474, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for HTTP/127.6.143.148@KRBTEST.COM
I20260811 11:57:55.446385 21281 consensus_queue.cc:1035] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bd73ab1bfd8447ca467c8568ed47394" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 36799 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:57:55.446626 21274 consensus_queue.cc:1035] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac [LEADER]: Connected to new peer: Peer: permanent_uuid: "24989c5801fc4f4d8b2a2c3c20515386" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 38515 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:57:55.458331 21286 mvcc.cc:205] Tried to move back new op lower bound from 7317297051415359488 to 7317297051195215872. Current Snapshot: MvccSnapshot[applied={T|T < 7317297051415359488}]
I20260811 11:57:55.469782 21285 mvcc.cc:205] Tried to move back new op lower bound from 7317297051415359488 to 7317297051195215872. Current Snapshot: MvccSnapshot[applied={T|T < 7317297051415359488}]
I20260811 11:57:55.480895 20711 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:57:55.481108 20711 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55846:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:57:55.484858 20711 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e25823411ce8459c8fa98e4f8d7b4174: Sending DeleteTablet for 3 replicas of tablet 94022030759c45eaac0ca14440bb8743
I20260811 11:57:55.485939 20995 tablet_service.cc:1514] Processing DeleteTablet for tablet 94022030759c45eaac0ca14440bb8743 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:57:55 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:35726
I20260811 11:57:55.486603 21152 tablet_service.cc:1514] Processing DeleteTablet for tablet 94022030759c45eaac0ca14440bb8743 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:57:55 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:32906
I20260811 11:57:55.487514 20835 tablet_service.cc:1514] Processing DeleteTablet for tablet 94022030759c45eaac0ca14440bb8743 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:57:55 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:35874
I20260811 11:57:55.488920 21295 tablet_replica.cc:330] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac: stopping tablet replica
I20260811 11:57:55.489166 21296 tablet_replica.cc:330] T 94022030759c45eaac0ca14440bb8743 P 24989c5801fc4f4d8b2a2c3c20515386: stopping tablet replica
I20260811 11:57:55.489563 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 20752
I20260811 11:57:55.491329 21295 raft_consensus.cc:2229] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:57:55.491765 21295 raft_consensus.cc:2258] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:57:55.493443 21295 ts_tablet_manager.cc:1905] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:57:55.496464 21295 ts_tablet_manager.cc:1918] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:57:55.496635 21295 log.cc:1196] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-2/wal/wals/94022030759c45eaac0ca14440bb8743
I20260811 11:57:55.497099 21295 ts_tablet_manager.cc:1939] T 94022030759c45eaac0ca14440bb8743 P c6e794b1469b486280bc99dedfda9fac: Deleting consensus metadata
I20260811 11:57:55.498436 20695 catalog_manager.cc:4841] TS c6e794b1469b486280bc99dedfda9fac (127.6.143.131:40805): tablet 94022030759c45eaac0ca14440bb8743 (table test-table [id=d4015880e922423890a0b08c80527ee2]) successfully deleted
I20260811 11:57:55.498796 21294 tablet_replica.cc:330] T 94022030759c45eaac0ca14440bb8743 P 6bd73ab1bfd8447ca467c8568ed47394: stopping tablet replica
I20260811 11:57:55.499171 21294 raft_consensus.cc:2229] T 94022030759c45eaac0ca14440bb8743 P 6bd73ab1bfd8447ca467c8568ed47394 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:57:55.499543 21294 raft_consensus.cc:2258] T 94022030759c45eaac0ca14440bb8743 P 6bd73ab1bfd8447ca467c8568ed47394 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:57:55.505820 21294 ts_tablet_manager.cc:1905] T 94022030759c45eaac0ca14440bb8743 P 6bd73ab1bfd8447ca467c8568ed47394: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:57:55.516685 21294 ts_tablet_manager.cc:1918] T 94022030759c45eaac0ca14440bb8743 P 6bd73ab1bfd8447ca467c8568ed47394: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:57:55.516819 21294 log.cc:1196] T 94022030759c45eaac0ca14440bb8743 P 6bd73ab1bfd8447ca467c8568ed47394: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786449319379472-6718-0/minicluster-data/ts-1/wal/wals/94022030759c45eaac0ca14440bb8743
I20260811 11:57:55.517195 21294 ts_tablet_manager.cc:1939] T 94022030759c45eaac0ca14440bb8743 P 6bd73ab1bfd8447ca467c8568ed47394: Deleting consensus metadata
I20260811 11:57:55.518620 20695 catalog_manager.cc:4841] TS 6bd73ab1bfd8447ca467c8568ed47394 (127.6.143.130:36799): tablet 94022030759c45eaac0ca14440bb8743 (table test-table [id=d4015880e922423890a0b08c80527ee2]) successfully deleted
W20260811 11:57:55.522122 20698 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.143.129:38515 (error 108)
I20260811 11:57:55.524130 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 20929
I20260811 11:57:55.524966 21263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:55.524596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:38515 (local address 127.0.0.1:35878)
0811 11:57:55.524797 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:55.524871 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.129:38515: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260811 11:57:55.525614 20698 catalog_manager.cc:4584] TS 24989c5801fc4f4d8b2a2c3c20515386 (127.6.143.129:38515): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 94022030759c45eaac0ca14440bb8743: Network error: Client connection negotiation failed: client connection to 127.6.143.129:38515: connect: Connection refused (error 111)
I20260811 11:57:55.547780 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 21087
I20260811 11:57:55.558104 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 20676
I20260811 11:57:55.570487 6718 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 11:57:55.915416 6718 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 11:57:55.915560 6718 mini_ranger.cc:66] Stopping Ranger...
I20260811 11:57:56.261591 6718 mini_ranger.cc:68] Stopped Ranger
2026-08-11 11:57:56.261 UTC [19649] LOG: received smart shutdown request
2026-08-11 11:57:56.265 UTC [19649] LOG: background worker "logical replication launcher" (PID 19658) exited with exit code 1
2026-08-11 11:57:56.266 UTC [19653] LOG: shutting down
2026-08-11 11:57:56.414 UTC [19649] LOG: database system is shut down
2026-08-11T11:57:56Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47504 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:57:56 dist-test-slave-zkpd krb5kdc[21309](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:57:56 dist-test-slave-zkpd krb5kdc[21309](info): set up 2 sockets
Aug 11 11:57:56 dist-test-slave-zkpd krb5kdc[21309](info): commencing operation
krb5kdc: starting...
W20260811 11:57:57.620903 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:57:57 dist-test-slave-zkpd krb5kdc[21309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:57:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:57:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
I20260811 11:57:57.742784 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:46115
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:34539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:46115,127.6.143.189:32855,127.6.143.188:34127
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:57:57.855398 21324 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:57:57.855646 21324 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:57:57.855686 21324 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:57:57.859556 21324 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:57:57.859611 21324 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:57:57.859629 21324 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:57:57.859645 21324 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:57:57.859660 21324 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:57:57.864749 21324 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:34539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:46115,127.6.143.189:32855,127.6.143.188:34127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:46115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21324
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:57:57.865658 21324 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:57:57.866569 21324 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:57:57.875037 21329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:57:57.875034 21330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:57:57.875093 21324 server_base.cc:1034] running on GCE node
W20260811 11:57:57.875034 21332 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:57:57.875816 21324 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:57:57.876860 21324 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:57:57.878050 21324 hybrid_clock.cc:648] HybridClock initialized: now 1786449477878021 us; error 37 us; skew 500 ppm
Aug 11 11:57:57 dist-test-slave-zkpd krb5kdc[21309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449477, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:57:57.880929 21324 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:57:57.881845 21324 webserver.cc:460] Webserver started at http://127.6.143.190:40343/ using document root <none> and password file <none>
I20260811 11:57:57.882396 21324 fs_manager.cc:359] Metadata directory not provided
I20260811 11:57:57.882491 21324 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:57:57.882733 21324 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:57:57.884796 21324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "ed480872227a4eb586ecf406c78d3ca1"
format_stamp: "Formatted at 2026-08-11 11:57:57 on dist-test-slave-zkpd"
server_key: "5c27ea68583b298f5ce4e648369f46ef"
server_key_iv: "1264bd0f86824adcfef3cd70706a5e26"
server_key_version: "encryptionkey@0"
I20260811 11:57:57.885353 21324 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "ed480872227a4eb586ecf406c78d3ca1"
format_stamp: "Formatted at 2026-08-11 11:57:57 on dist-test-slave-zkpd"
server_key: "5c27ea68583b298f5ce4e648369f46ef"
server_key_iv: "1264bd0f86824adcfef3cd70706a5e26"
server_key_version: "encryptionkey@0"
I20260811 11:57:57.888927 21324 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 11:57:57.891355 21341 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:57.892683 21324 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:57:57.892845 21324 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "ed480872227a4eb586ecf406c78d3ca1"
format_stamp: "Formatted at 2026-08-11 11:57:57 on dist-test-slave-zkpd"
server_key: "5c27ea68583b298f5ce4e648369f46ef"
server_key_iv: "1264bd0f86824adcfef3cd70706a5e26"
server_key_version: "encryptionkey@0"
I20260811 11:57:57.892966 21324 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:57:57.915957 21324 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:57:57.918648 21324 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:57:57.918849 21324 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:57:57.926077 21324 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:46115
I20260811 11:57:57.926103 21394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:46115 every 8 connection(s)
I20260811 11:57:57.927002 21324 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:57:57.929243 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 21324
I20260811 11:57:57.929378 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:57:57.929673 6718 external_mini_cluster.cc:1428] Setting key 760dc042721103a576cecc621cb56cc5
I20260811 11:57:57.929838 21396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:57:57.933560 21396 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46115 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.143.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.189@KRBTEST.COM" created.
Aug 11 11:57:57 dist-test-slave-zkpd krb5kdc[21309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449477, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:57:57.948356 21400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:57.935744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47222 (local address 127.6.143.190:46115)
0811 11:57:57.936340 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:57.936352 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:57.936398 (+ 46us) server_negotiation.cc:407] Connection header received
0811 11:57:57.937402 (+ 1004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:57.937408 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:57.937660 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:57.937875 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:57:57.938618 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:57.939700 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:57.940715 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:57.941087 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:57.941199 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:57.943619 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:57:57.943625 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:57:57.943635 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:57:57.943655 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:57:57.945293 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:57.945786 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:57.945792 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:57.945794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:57.945905 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:57.946229 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:57.946233 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:57.946234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:57.946589 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:57:57.946734 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:57.947093 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 11:57:57.947334 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":200,"threads_started":1}
I20260811 11:57:57.948355 21399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:57.935744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46115 (local address 127.0.0.1:47222)
0811 11:57:57.936233 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:57.936284 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 11:57:57.937078 (+ 794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:57:57.937867 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:57:57.937871 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:57:57.938174 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:57:57.938464 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:57.938477 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:57.939854 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:57.939858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:57.940608 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:57.940618 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:57.941226 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:57.941229 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:57.941406 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:57.942023 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:57:57.942039 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:57:57.943466 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:57:57.945429 (+ 1963us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:57.945439 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:57.945449 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:57.945672 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:57.946010 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:57.946013 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:57.946015 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:57.946129 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:57.946706 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:57:57.946712 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:57:57.946855 (+ 143us) client_negotiation.cc:769] Sending connection context
0811 11:57:57.947030 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 11:57:57.947419 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":108,"thread_start_us":179,"threads_started":1}
I20260811 11:57:57.951085 21396 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.189" port: 32855 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:57:57.951804 21399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:57.951557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:32855 (local address 127.0.0.1:39172)
0811 11:57:57.951681 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:57.951746 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.189:32855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 11:57:57.952020 21345 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.143.189:32855: connect: Connection refused (error 111)
I20260811 11:57:57.952924 21399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:57.952724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:32855 (local address 127.0.0.1:39178)
0811 11:57:57.952826 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:57.952870 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.189:32855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 11:57:57.953136 21396 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.143.189:32855: Network error: Client connection negotiation failed: client connection to 127.6.143.189:32855: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.189.keytab.
Entry for principal kudu/127.6.143.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.189.keytab.
I20260811 11:57:57.956728 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.189:32855
--webserver_interface=127.6.143.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.189.keytab
--principal=kudu/127.6.143.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:34539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:46115,127.6.143.189:32855,127.6.143.188:34127
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:57:58.002396 21396 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.143.189" port: 32855 } attempt: 1
I20260811 11:57:58.003280 21399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.002975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:32855 (local address 127.0.0.1:39190)
0811 11:57:58.003130 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:58.003201 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.189:32855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260811 11:57:58.004098 21399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.003835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:32855 (local address 127.0.0.1:39204)
0811 11:57:58.003976 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:58.004035 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.189:32855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 11:57:58.004257 21396 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.143.189:32855: Network error: Client connection negotiation failed: client connection to 127.6.143.189:32855: connect: Connection refused (error 111)
W20260811 11:57:58.069445 21404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:57:58.069679 21404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:57:58.069717 21404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 11:57:58.072405 21396 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.143.189" port: 32855 } attempt: 2
I20260811 11:57:58.073148 21399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.072869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:32855 (local address 127.0.0.1:39214)
0811 11:57:58.073010 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:58.073078 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.189:32855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 11:57:58.073836 21404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:57:58.073884 21404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:57:58.073902 21404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:57:58.073937 21404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
I20260811 11:57:58.073942 21399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.073717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:32855 (local address 127.0.0.1:39228)
0811 11:57:58.073826 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:58.073873 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.189:32855: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 11:57:58.073972 21404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 11:57:58.074090 21396 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.143.189:32855: Network error: Client connection negotiation failed: client connection to 127.6.143.189:32855: connect: Connection refused (error 111)
I20260811 11:57:58.078940 21404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:34539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:46115,127.6.143.189:32855,127.6.143.188:34127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.189.keytab
--principal=kudu/127.6.143.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.189:32855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.143.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21404
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:57:58.079797 21404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:57:58.080734 21404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:57:58.088864 21411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:57:58.088888 21414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:57:58.088960 21409 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:57:58.089164 21404 server_base.cc:1034] running on GCE node
I20260811 11:57:58.089761 21404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:57:58.090803 21404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:57:58.091961 21404 hybrid_clock.cc:648] HybridClock initialized: now 1786449478091948 us; error 24 us; skew 500 ppm
Aug 11 11:57:58 dist-test-slave-zkpd krb5kdc[21309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449478, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:57:58.094635 21404 init.cc:377] Logged in from keytab as kudu/127.6.143.189@KRBTEST.COM (short username kudu)
I20260811 11:57:58.095513 21404 webserver.cc:460] Webserver started at http://127.6.143.189:41263/ using document root <none> and password file <none>
I20260811 11:57:58.096131 21404 fs_manager.cc:359] Metadata directory not provided
I20260811 11:57:58.096218 21404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:57:58.096465 21404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:57:58.098461 21404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-1/data/instance:
uuid: "e1a2eb5dd9494f0387bec664455ca098"
format_stamp: "Formatted at 2026-08-11 11:57:58 on dist-test-slave-zkpd"
server_key: "488dfef62f3e6031c83e308d3624cd25"
server_key_iv: "4eeea88285b8d123d89ce5b697a9a5cc"
server_key_version: "encryptionkey@0"
I20260811 11:57:58.099014 21404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-1/wal/instance:
uuid: "e1a2eb5dd9494f0387bec664455ca098"
format_stamp: "Formatted at 2026-08-11 11:57:58 on dist-test-slave-zkpd"
server_key: "488dfef62f3e6031c83e308d3624cd25"
server_key_iv: "4eeea88285b8d123d89ce5b697a9a5cc"
server_key_version: "encryptionkey@0"
I20260811 11:57:58.102669 21404 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 11:57:58.105171 21423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:58.106348 21404 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 11:57:58.106474 21404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-1/wal
uuid: "e1a2eb5dd9494f0387bec664455ca098"
format_stamp: "Formatted at 2026-08-11 11:57:58 on dist-test-slave-zkpd"
server_key: "488dfef62f3e6031c83e308d3624cd25"
server_key_iv: "4eeea88285b8d123d89ce5b697a9a5cc"
server_key_version: "encryptionkey@0"
I20260811 11:57:58.106568 21404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:57:58.132789 21404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:57:58.135824 21404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:57:58.136062 21404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:57:58.143323 21404 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.189:32855
I20260811 11:57:58.143396 21480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.189:32855 every 8 connection(s)
I20260811 11:57:58.144301 21404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-1/data/info.pb
I20260811 11:57:58.147137 21481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:57:58.150913 21481 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46115 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:57:58.153578 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 21404
I20260811 11:57:58.153734 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-1/wal/instance
I20260811 11:57:58.154034 6718 external_mini_cluster.cc:1428] Setting key 62a7d4dc05144a1be2141aa71c0ee70f
Aug 11 11:57:58 dist-test-slave-zkpd krb5kdc[21309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449478, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.189@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.143.188@KRBTEST.COM; defaulting to no policy
I20260811 11:57:58.165275 21400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.152742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47226 (local address 127.6.143.190:46115)
0811 11:57:58.152894 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:58.152898 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:58.153668 (+ 770us) server_negotiation.cc:407] Connection header received
0811 11:57:58.154673 (+ 1005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:58.154677 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:58.154739 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:58.154842 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:57:58.156147 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.156687 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.158045 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.158281 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.158396 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:58.160952 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:57:58.160955 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:57:58.160957 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:57:58.160981 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:57:58.163109 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:58.163631 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:58.163639 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:58.163640 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:58.163714 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:58.164129 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:58.164133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:58.164135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:58.164321 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:57:58.164436 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:58.164972 (+ 536us) server_negotiation.cc:299] Negotiation successful
0811 11:57:58.165101 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 11:57:58.165262 21396 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.143.189" port: 32855 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.188@KRBTEST.COM" created.
I20260811 11:57:58.166236 21483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.153082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46115 (local address 127.0.0.1:47226)
0811 11:57:58.153521 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:58.153564 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 11:57:58.154463 (+ 899us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:57:58.155016 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:57:58.155023 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:57:58.155355 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:57:58.155977 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:58.156001 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.156830 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.156834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:58.157897 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:58.157909 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.158445 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.158449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:58.158626 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:58.159324 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:57:58.159339 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:57:58.160768 (+ 1429us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:57:58.163229 (+ 2461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:58.163235 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:58.163250 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:58.163516 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:58.163809 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:58.163813 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:58.163815 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:58.164029 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:58.164442 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:57:58.164450 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:57:58.164705 (+ 255us) client_negotiation.cc:769] Sending connection context
0811 11:57:58.164890 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 11:57:58.165218 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":141,"threads_started":1}
I20260811 11:57:58.167936 21481 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.189" port: 32855 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 11:57:58 dist-test-slave-zkpd krb5kdc[21309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449477, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for kudu/127.6.143.189@KRBTEST.COM
Aug 11 11:57:58 dist-test-slave-zkpd krb5kdc[21309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449478, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.189@KRBTEST.COM for kudu/127.6.143.189@KRBTEST.COM
I20260811 11:57:58.176797 21399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.165752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:32855 (local address 127.0.0.1:39234)
0811 11:57:58.166094 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:58.166108 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:57:58.166192 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:57:58.166692 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:57:58.166695 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:57:58.166868 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:57:58.166977 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:58.166983 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.168283 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.168288 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:58.169138 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:58.169147 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.169816 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.169819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:58.169936 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:58.170476 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:57:58.170490 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:57:58.172182 (+ 1692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:57:58.174452 (+ 2270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:58.174456 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:58.174458 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:58.174655 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:58.175522 (+ 867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:58.175525 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:58.175527 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:58.175630 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:58.176162 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:57:58.176166 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:57:58.176305 (+ 139us) client_negotiation.cc:769] Sending connection context
0811 11:57:58.176451 (+ 146us) client_negotiation.cc:240] Negotiation successful
0811 11:57:58.176594 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261}
I20260811 11:57:58.176920 21485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.165868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39234 (local address 127.6.143.189:32855)
0811 11:57:58.166279 (+ 411us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:58.166292 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:58.166318 (+ 26us) server_negotiation.cc:407] Connection header received
0811 11:57:58.166359 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:58.166365 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:58.166561 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:58.166711 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:57:58.167104 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.168174 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.169256 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.169686 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.169802 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:58.172384 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:57:58.172393 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:57:58.172403 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:57:58.172424 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:57:58.174336 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:58.174769 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:58.174775 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:58.174777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:58.175420 (+ 643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:58.175735 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:58.175737 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:58.175739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:58.176043 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:57:58.176199 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:58.176559 (+ 360us) server_negotiation.cc:299] Negotiation successful
0811 11:57:58.176739 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":16,"server-negotiator.queue_time_us":314,"thread_start_us":139,"threads_started":1}
I20260811 11:57:58.178123 21487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.168504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39248 (local address 127.6.143.189:32855)
0811 11:57:58.168770 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:58.168778 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:58.168790 (+ 12us) server_negotiation.cc:407] Connection header received
0811 11:57:58.168825 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:58.168827 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:58.168866 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:58.168957 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:57:58.169563 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.170486 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.171411 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.171658 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.171767 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:58.174513 (+ 2746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:57:58.174519 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:57:58.174524 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:57:58.174551 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:57:58.176378 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:58.176843 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:58.176848 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:58.176850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:58.176943 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:58.177256 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:58.177259 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:58.177261 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:58.177465 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:57:58.177563 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:58.177829 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 11:57:58.177970 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":207,"thread_start_us":107,"threads_started":1}
I20260811 11:57:58.178129 21483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.168439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:32855 (local address 127.0.0.1:39248)
0811 11:57:58.168615 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:58.168632 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 11:57:58.168707 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:57:58.168999 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:57:58.169004 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:57:58.169191 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:57:58.169429 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:58.169442 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.170613 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.170616 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:58.171283 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:58.171294 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.171767 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.171771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:58.171863 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:58.172645 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:57:58.172655 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:57:58.174329 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:57:58.176489 (+ 2160us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:58.176495 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:58.176498 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:58.176757 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:58.177053 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:58.177057 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:58.177061 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:58.177129 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:58.177585 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:57:58.177587 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:57:58.177666 (+ 79us) client_negotiation.cc:769] Sending connection context
0811 11:57:58.177783 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 11:57:58.177914 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":63}
I20260811 11:57:58.178831 21481 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.188" port: 34127 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:57:58.178877 21396 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.188" port: 34127 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:57:58.179579 21399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.179330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:34127 (local address 127.0.0.1:54578)
0811 11:57:58.179478 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:58.179521 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:34127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260811 11:57:58.179566 21483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.179267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:34127 (local address 127.0.0.1:54562)
0811 11:57:58.179394 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:58.179476 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:34127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 11:57:58.179854 21427 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.143.188:34127: connect: Connection refused (error 111)
I20260811 11:57:58.180248 21399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.179980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:34127 (local address 127.0.0.1:54590)
0811 11:57:58.180123 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:58.180173 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:34127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 11:57:58.180387 21396 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.143.188:34127: Network error: Client connection negotiation failed: client connection to 127.6.143.188:34127: connect: Connection refused (error 111)
I20260811 11:57:58.180797 21483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.180630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:34127 (local address 127.0.0.1:54596)
0811 11:57:58.180711 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:58.180752 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:34127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260811 11:57:58.180931 21481 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.143.188:34127: Network error: Client connection negotiation failed: client connection to 127.6.143.188:34127: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.188.keytab.
Entry for principal kudu/127.6.143.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.188.keytab.
I20260811 11:57:58.183252 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.188:34127
--webserver_interface=127.6.143.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.188.keytab
--principal=kudu/127.6.143.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:34539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:46115,127.6.143.189:32855,127.6.143.188:34127
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:57:58.211542 21396 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.143.188" port: 34127 } attempt: 1
I20260811 11:57:58.212446 21399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.212139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:34127 (local address 127.0.0.1:54612)
0811 11:57:58.212293 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:58.212363 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:34127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 11:57:58.213167 21399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.212952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:34127 (local address 127.0.0.1:54624)
0811 11:57:58.213064 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:58.213115 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:34127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 11:57:58.213299 21396 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.143.188:34127: Network error: Client connection negotiation failed: client connection to 127.6.143.188:34127: connect: Connection refused (error 111)
I20260811 11:57:58.230310 21481 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.143.188" port: 34127 } attempt: 1
I20260811 11:57:58.231143 21483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.230868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:34127 (local address 127.0.0.1:54632)
0811 11:57:58.231010 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:58.231071 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:34127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 11:57:58.231877 21483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.231660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:34127 (local address 127.0.0.1:54648)
0811 11:57:58.231771 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:58.231822 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:34127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 11:57:58.232065 21481 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.143.188:34127: Network error: Client connection negotiation failed: client connection to 127.6.143.188:34127: connect: Connection refused (error 111)
I20260811 11:57:58.288479 21396 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.143.188" port: 34127 } attempt: 2
I20260811 11:57:58.289318 21399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.289013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:34127 (local address 127.0.0.1:54654)
0811 11:57:58.289170 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:58.289241 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:34127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260811 11:57:58.290046 21399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.289821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:34127 (local address 127.0.0.1:54658)
0811 11:57:58.289949 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:58.289999 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:34127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 11:57:58.290176 21396 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.143.188:34127: Network error: Client connection negotiation failed: client connection to 127.6.143.188:34127: connect: Connection refused (error 111)
W20260811 11:57:58.297780 21489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:57:58.298182 21489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:57:58.298262 21489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 11:57:58.300247 21481 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.143.188" port: 34127 } attempt: 2
I20260811 11:57:58.301038 21483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.300763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:34127 (local address 127.0.0.1:54674)
0811 11:57:58.300901 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:58.300963 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:34127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 11:57:58.301838 21483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.301630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:34127 (local address 127.0.0.1:54680)
0811 11:57:58.301735 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:58.301784 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:34127: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 11:57:58.302024 21481 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.143.188:34127: Network error: Client connection negotiation failed: client connection to 127.6.143.188:34127: connect: Connection refused (error 111)
W20260811 11:57:58.303049 21489 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:57:58.303100 21489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:57:58.303120 21489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:57:58.303172 21489 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:57:58.303208 21489 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:57:58.308609 21489 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:34539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:46115,127.6.143.189:32855,127.6.143.188:34127
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.188.keytab
--principal=kudu/127.6.143.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.188:34127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.143.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:57:58.309711 21489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:57:58.310849 21489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:57:58.319242 21494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:57:58.319314 21497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:57:58.319212 21499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:57:58.319391 21489 server_base.cc:1034] running on GCE node
I20260811 11:57:58.320179 21489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:57:58.321228 21489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:57:58.322392 21489 hybrid_clock.cc:648] HybridClock initialized: now 1786449478322380 us; error 29 us; skew 500 ppm
Aug 11 11:57:58 dist-test-slave-zkpd krb5kdc[21309](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449478, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:57:58.325193 21489 init.cc:377] Logged in from keytab as kudu/127.6.143.188@KRBTEST.COM (short username kudu)
I20260811 11:57:58.326032 21489 webserver.cc:460] Webserver started at http://127.6.143.188:35969/ using document root <none> and password file <none>
I20260811 11:57:58.326583 21489 fs_manager.cc:359] Metadata directory not provided
I20260811 11:57:58.326645 21489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:57:58.326890 21489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:57:58.328943 21489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-2/data/instance:
uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc"
format_stamp: "Formatted at 2026-08-11 11:57:58 on dist-test-slave-zkpd"
server_key: "771e23f46344a7379c800a99440a2a0c"
server_key_iv: "22a7e386cd751516ba72cd9651fc64de"
server_key_version: "encryptionkey@0"
I20260811 11:57:58.329500 21489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-2/wal/instance:
uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc"
format_stamp: "Formatted at 2026-08-11 11:57:58 on dist-test-slave-zkpd"
server_key: "771e23f46344a7379c800a99440a2a0c"
server_key_iv: "22a7e386cd751516ba72cd9651fc64de"
server_key_version: "encryptionkey@0"
I20260811 11:57:58.333078 21489 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 11:57:58.335413 21506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:57:58.336623 21489 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:57:58.336764 21489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-2/wal
uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc"
format_stamp: "Formatted at 2026-08-11 11:57:58 on dist-test-slave-zkpd"
server_key: "771e23f46344a7379c800a99440a2a0c"
server_key_iv: "22a7e386cd751516ba72cd9651fc64de"
server_key_version: "encryptionkey@0"
I20260811 11:57:58.336879 21489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:57:58.355093 21489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:57:58.357829 21489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:57:58.358047 21489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:57:58.365446 21489 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.188:34127
I20260811 11:57:58.365473 21564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.188:34127 every 8 connection(s)
I20260811 11:57:58.366389 21489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-2/data/info.pb
I20260811 11:57:58.369256 21565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:57:58.370213 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 21489
I20260811 11:57:58.370366 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786449319379472-6718-0/minicluster-data/master-2/wal/instance
I20260811 11:57:58.370692 6718 external_mini_cluster.cc:1428] Setting key 5d3409de496e8d1db6aa20b36e200026
I20260811 11:57:58.373662 21565 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46115 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 11:57:58 dist-test-slave-zkpd krb5kdc[21309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.188@KRBTEST.COM
Aug 11 11:57:58 dist-test-slave-zkpd krb5kdc[21309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449478, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.188@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:57:58.384533 21400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.375358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47228 (local address 127.6.143.190:46115)
0811 11:57:58.375501 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:58.375507 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:58.375802 (+ 295us) server_negotiation.cc:407] Connection header received
0811 11:57:58.376049 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:58.376051 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:58.376099 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:58.376143 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:57:58.376888 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.377369 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.378406 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.378622 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.378736 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:58.381194 (+ 2458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:57:58.381197 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:57:58.381200 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:57:58.381229 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:57:58.382630 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:58.383145 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:58.383148 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:58.383150 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:58.383233 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:58.383506 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:58.383509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:58.383511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:58.383742 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:57:58.383832 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:58.384246 (+ 414us) server_negotiation.cc:299] Negotiation successful
0811 11:57:58.384359 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 11:57:58.385264 21569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.375288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46115 (local address 127.0.0.1:47228)
0811 11:57:58.375674 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:58.375713 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 11:57:58.375863 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:57:58.376300 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:57:58.376304 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:57:58.376639 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:57:58.376753 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:58.376758 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.377496 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.377500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:58.378266 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:58.378280 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.378774 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.378778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:58.378889 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:58.379391 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:57:58.379413 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:57:58.381026 (+ 1613us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:57:58.382736 (+ 1710us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:58.382744 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:58.382748 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:58.383051 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:58.383325 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:58.383328 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:58.383329 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:58.383414 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:58.383844 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:57:58.383851 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:57:58.384024 (+ 173us) client_negotiation.cc:769] Sending connection context
0811 11:57:58.384181 (+ 157us) client_negotiation.cc:240] Negotiation successful
0811 11:57:58.384552 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"mutex_wait_us":91,"thread_start_us":149,"threads_started":1}
I20260811 11:57:58.385269 21568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.372097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54694 (local address 127.6.143.188:34127)
0811 11:57:58.372545 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:58.372560 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:58.372595 (+ 35us) server_negotiation.cc:407] Connection header received
0811 11:57:58.373216 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:58.373232 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:58.373560 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:58.373909 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:57:58.374730 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.376174 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.377032 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.377506 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.377683 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:58.380194 (+ 2511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:57:58.380199 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:57:58.380209 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:57:58.380230 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:57:58.382219 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:58.382637 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:58.382643 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:58.382650 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:58.382814 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:58.383144 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:58.383147 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:58.383149 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:58.383650 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:57:58.383852 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:58.384134 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 11:57:58.384468 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":311,"spinlock_wait_cycles":768,"thread_start_us":135,"threads_started":1}
I20260811 11:57:58.386549 21565 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.189" port: 32855 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:57:58.389328 21396 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.143.188" port: 34127 } attempt: 3
Aug 11 11:57:58 dist-test-slave-zkpd krb5kdc[21309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449478, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.188@KRBTEST.COM for kudu/127.6.143.189@KRBTEST.COM
Aug 11 11:57:58 dist-test-slave-zkpd krb5kdc[21309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:57:58.393186 21481 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.143.188" port: 34127 } attempt: 3
Aug 11 11:57:58 dist-test-slave-zkpd krb5kdc[21309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449477, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for kudu/127.6.143.188@KRBTEST.COM
I20260811 11:57:58.396004 21487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.387168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39256 (local address 127.6.143.189:32855)
0811 11:57:58.387295 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:58.387300 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:58.387314 (+ 14us) server_negotiation.cc:407] Connection header received
0811 11:57:58.387370 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:58.387374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:58.387431 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:58.387498 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:57:58.387956 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.388481 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.389223 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.389525 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.389636 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:58.391984 (+ 2348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:57:58.391989 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:57:58.391994 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:57:58.392027 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:57:58.393751 (+ 1724us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:58.394225 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:58.394229 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:58.394230 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:58.394336 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:58.394881 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:58.394886 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:58.394889 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:58.395149 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:57:58.395258 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:58.395617 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 11:57:58.395743 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 11:57:58.396504 21569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.387006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:32855 (local address 127.0.0.1:39256)
0811 11:57:58.387179 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:58.387193 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:57:58.387262 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:57:58.387524 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:57:58.387526 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:57:58.387686 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:57:58.387811 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:58.387817 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.388594 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.388600 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:58.389099 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:58.389106 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.389646 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.389649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:58.389743 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:58.390418 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:57:58.390427 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:57:58.391762 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:57:58.393865 (+ 2103us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:58.393868 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:58.393870 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:58.394123 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:58.394668 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:58.394674 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:58.394677 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:58.394771 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:58.395308 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:57:58.395311 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:57:58.395405 (+ 94us) client_negotiation.cc:769] Sending connection context
0811 11:57:58.396186 (+ 781us) client_negotiation.cc:240] Negotiation successful
0811 11:57:58.396337 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":3}
I20260811 11:57:58.397640 21565 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.188" port: 34127 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:57:58.398036 21400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.387689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47232 (local address 127.6.143.190:46115)
0811 11:57:58.387798 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:58.387802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:58.387815 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:57:58.387989 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:58.387993 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:58.388042 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:58.388125 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:57:58.388724 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.389186 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.390188 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.390463 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.390565 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:58.393799 (+ 3234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:57:58.393804 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:57:58.393810 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:57:58.393839 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:57:58.395761 (+ 1922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:58.396338 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:58.396342 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:58.396343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:58.396434 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:58.396761 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:58.396765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:58.396766 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:58.397001 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:57:58.397194 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:58.397771 (+ 577us) server_negotiation.cc:299] Negotiation successful
0811 11:57:58.397890 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Aug 11 11:57:58 dist-test-slave-zkpd krb5kdc[21309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449478, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.189@KRBTEST.COM for kudu/127.6.143.188@KRBTEST.COM
I20260811 11:57:58.400281 21399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.389669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:34127 (local address 127.0.0.1:54696)
0811 11:57:58.389881 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:58.389893 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:57:58.389958 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:57:58.390391 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:57:58.390394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:57:58.390577 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:57:58.390689 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:58.390695 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.392197 (+ 1502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.392202 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:58.392876 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:58.392888 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.393428 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.393432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:58.393523 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:58.393949 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:57:58.393964 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:57:58.395580 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:57:58.398127 (+ 2547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:58.398130 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:58.398132 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:58.398378 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:58.398963 (+ 585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:58.398967 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:58.398969 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:58.399064 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:58.399706 (+ 642us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:57:58.399710 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:57:58.399814 (+ 104us) client_negotiation.cc:769] Sending connection context
0811 11:57:58.399991 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 11:57:58.400121 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140}
I20260811 11:57:58.400393 21568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.390077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54696 (local address 127.6.143.188:34127)
0811 11:57:58.390197 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:58.390201 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:58.390213 (+ 12us) server_negotiation.cc:407] Connection header received
0811 11:57:58.390249 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:58.390251 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:58.390281 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:58.390343 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:57:58.391015 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.392087 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.393027 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.393269 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.393744 (+ 475us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:58.395727 (+ 1983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:57:58.395731 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:57:58.395733 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:57:58.395752 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:57:58.398004 (+ 2252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:58.398762 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:58.398768 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:58.398773 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:58.398851 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:58.399171 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:58.399174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:58.399177 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:58.399575 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:57:58.399706 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:58.400050 (+ 344us) server_negotiation.cc:299] Negotiation successful
0811 11:57:58.400232 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":47}
Aug 11 11:57:58 dist-test-slave-zkpd krb5kdc[21309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449478, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.188@KRBTEST.COM for kudu/127.6.143.188@KRBTEST.COM
I20260811 11:57:58.404911 21483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.393564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:34127 (local address 127.0.0.1:54712)
0811 11:57:58.393987 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:58.394004 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 11:57:58.394391 (+ 387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:57:58.395313 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:57:58.395316 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:57:58.395505 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:57:58.395606 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:58.395616 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.397010 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.397014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:58.397781 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:58.397792 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.398418 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.398422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:58.398507 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:58.398924 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:57:58.398938 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:57:58.400437 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:57:58.403238 (+ 2801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:58.403241 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:58.403243 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:58.403516 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:58.403841 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:58.403844 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:58.403846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:58.403961 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:58.404405 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:57:58.404408 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:57:58.404499 (+ 91us) client_negotiation.cc:769] Sending connection context
0811 11:57:58.404580 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 11:57:58.404718 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344}
I20260811 11:57:58.404600 21396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed480872227a4eb586ecf406c78d3ca1: Bootstrap starting.
I20260811 11:57:58.405866 21571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.394428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54712 (local address 127.6.143.188:34127)
0811 11:57:58.395053 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:58.395064 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:58.395079 (+ 15us) server_negotiation.cc:407] Connection header received
0811 11:57:58.395118 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:58.395121 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:58.395164 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:58.395716 (+ 552us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:57:58.395799 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.396859 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.397927 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.398272 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.398413 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:58.400598 (+ 2185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:57:58.400603 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:57:58.400606 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:57:58.400637 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:57:58.403114 (+ 2477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:58.403643 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:58.403646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:58.403647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:58.403732 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:58.404083 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:58.404085 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:58.404087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:58.404272 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:57:58.405525 (+ 1253us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:58.405603 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 11:57:58.405710 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":121,"threads_started":1}
Aug 11 11:57:58 dist-test-slave-zkpd krb5kdc[21309](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449477, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.189@KRBTEST.COM
I20260811 11:57:58.407851 21569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.398137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:34127 (local address 127.0.0.1:54714)
0811 11:57:58.398653 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0811 11:57:58.398666 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:57:58.398741 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:57:58.399410 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:57:58.399413 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:57:58.399568 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:57:58.399667 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:58.399676 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.400879 (+ 1203us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.400881 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:58.401638 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:57:58.401650 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.402239 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.402243 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:57:58.402323 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:58.402771 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:57:58.402789 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:57:58.404257 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:57:58.406288 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:58.406291 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:58.406294 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:58.406553 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:58.406836 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:57:58.406839 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:57:58.406841 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:57:58.406912 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:57:58.407346 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:57:58.407349 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:57:58.407445 (+ 96us) client_negotiation.cc:769] Sending connection context
0811 11:57:58.407549 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 11:57:58.407674 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455}
I20260811 11:57:58.408491 21572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.398336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54714 (local address 127.6.143.188:34127)
0811 11:57:58.399085 (+ 749us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:58.399096 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:58.399110 (+ 14us) server_negotiation.cc:407] Connection header received
0811 11:57:58.399146 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:58.399151 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:58.399190 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:58.399268 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:57:58.399772 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.400763 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.401768 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.402118 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.402223 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:58.404375 (+ 2152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:57:58.404378 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:57:58.404380 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:57:58.404410 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:57:58.406171 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:58.406663 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:58.406667 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:58.406669 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:58.406747 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:58.407002 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:58.407007 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:58.407011 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:58.407211 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:57:58.407310 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:58.408169 (+ 859us) server_negotiation.cc:299] Negotiation successful
0811 11:57:58.408281 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":690,"thread_start_us":102,"threads_started":1}
I20260811 11:57:58.408844 21396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed480872227a4eb586ecf406c78d3ca1: Neither blocks nor log segments found. Creating new log.
I20260811 11:57:58.409245 21481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1a2eb5dd9494f0387bec664455ca098: Bootstrap starting.
I20260811 11:57:58.409940 21396 log.cc:824] T 00000000000000000000000000000000 P ed480872227a4eb586ecf406c78d3ca1: Log is configured to *not* fsync() on all Append() calls
I20260811 11:57:58.411368 21487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.399472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39270 (local address 127.6.143.189:32855)
0811 11:57:58.399608 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:58.399614 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:58.401062 (+ 1448us) server_negotiation.cc:407] Connection header received
0811 11:57:58.401199 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:58.401201 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:58.401243 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:58.401314 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:57:58.402071 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.402758 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.404157 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.404484 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.404569 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:58.407592 (+ 3023us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:57:58.407596 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:57:58.407598 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:57:58.407622 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:57:58.409260 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:58.409748 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:58.409752 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:58.409753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:58.409839 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:58.410156 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:58.410159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:58.410161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:58.410372 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:57:58.411126 (+ 754us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:58.411189 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 11:57:58.411273 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 11:57:58.411782 21481 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e1a2eb5dd9494f0387bec664455ca098: Neither blocks nor log segments found. Creating new log.
I20260811 11:57:58.412364 21396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed480872227a4eb586ecf406c78d3ca1: No bootstrap required, opened a new log
I20260811 11:57:58.412989 21481 log.cc:824] T 00000000000000000000000000000000 P e1a2eb5dd9494f0387bec664455ca098: Log is configured to *not* fsync() on all Append() calls
I20260811 11:57:58.414454 21565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc: Bootstrap starting.
I20260811 11:57:58.414624 6718 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:57:58.416491 21565 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc: Neither blocks nor log segments found. Creating new log.
I20260811 11:57:58.417069 21396 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed480872227a4eb586ecf406c78d3ca1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed480872227a4eb586ecf406c78d3ca1" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46115 } } peers { permanent_uuid: "e1a2eb5dd9494f0387bec664455ca098" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 32855 } } peers { permanent_uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 34127 } }
I20260811 11:57:58.417296 21396 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed480872227a4eb586ecf406c78d3ca1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:57:58.417291 21565 log.cc:824] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc: Log is configured to *not* fsync() on all Append() calls
I20260811 11:57:58.417351 21396 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed480872227a4eb586ecf406c78d3ca1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed480872227a4eb586ecf406c78d3ca1, State: Initialized, Role: FOLLOWER
I20260811 11:57:58.417889 21396 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed480872227a4eb586ecf406c78d3ca1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed480872227a4eb586ecf406c78d3ca1" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46115 } } peers { permanent_uuid: "e1a2eb5dd9494f0387bec664455ca098" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 32855 } } peers { permanent_uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 34127 } }
I20260811 11:57:58.418422 21585 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed480872227a4eb586ecf406c78d3ca1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed480872227a4eb586ecf406c78d3ca1" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46115 } } peers { permanent_uuid: "e1a2eb5dd9494f0387bec664455ca098" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 32855 } } peers { permanent_uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 34127 } } }
I20260811 11:57:58.418644 21585 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed480872227a4eb586ecf406c78d3ca1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:57:58.419376 21396 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed480872227a4eb586ecf406c78d3ca1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:57:58.419834 21565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc: No bootstrap required, opened a new log
W20260811 11:57:58.420328 21347 tablet.cc:2369] T 00000000000000000000000000000000 P ed480872227a4eb586ecf406c78d3ca1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:57:58.423959 21481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e1a2eb5dd9494f0387bec664455ca098: No bootstrap required, opened a new log
I20260811 11:57:58.424974 21565 raft_consensus.cc:348] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed480872227a4eb586ecf406c78d3ca1" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46115 } } peers { permanent_uuid: "e1a2eb5dd9494f0387bec664455ca098" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 32855 } } peers { permanent_uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 34127 } }
I20260811 11:57:58.425262 21565 raft_consensus.cc:374] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:57:58.425323 21565 raft_consensus.cc:729] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8812db2a37b49b1b5b4ff5e074b1cbc, State: Initialized, Role: FOLLOWER
I20260811 11:57:58.427214 21565 consensus_queue.cc:260] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed480872227a4eb586ecf406c78d3ca1" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46115 } } peers { permanent_uuid: "e1a2eb5dd9494f0387bec664455ca098" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 32855 } } peers { permanent_uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 34127 } }
I20260811 11:57:58.428758 21565 sys_catalog.cc:564] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:57:58.431161 21487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.417352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39284 (local address 127.6.143.189:32855)
0811 11:57:58.417475 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:58.417479 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:58.419206 (+ 1727us) server_negotiation.cc:407] Connection header received
0811 11:57:58.419385 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:58.419390 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:58.419445 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:58.419504 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:57:58.420491 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.421168 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.422352 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.422665 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.422748 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:58.427126 (+ 4378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:57:58.427130 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:57:58.427133 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:57:58.427158 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:57:58.428800 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:58.429609 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:58.429616 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:58.429618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:58.429716 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:58.430096 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:58.430102 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:58.430104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:58.430342 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:57:58.430458 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:58.430865 (+ 407us) server_negotiation.cc:299] Negotiation successful
0811 11:57:58.431008 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260811 11:57:58.434808 21611 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 11:57:58.435360 21611 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 11:57:58.435472 21571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.417814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54730 (local address 127.6.143.188:34127)
0811 11:57:58.417911 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:58.417916 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:58.425977 (+ 8061us) server_negotiation.cc:407] Connection header received
0811 11:57:58.426154 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:58.426158 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:58.426204 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:58.426254 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:57:58.427111 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.427941 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.429021 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.429381 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.429459 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:58.431130 (+ 1671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:57:58.431136 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:57:58.431142 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:57:58.431176 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:57:58.433029 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:58.433535 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:58.433539 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:58.433540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:58.433629 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:58.433977 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:58.433981 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:58.433982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:58.434186 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:57:58.434340 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:58.434966 (+ 626us) server_negotiation.cc:299] Negotiation successful
0811 11:57:58.435082 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 11:57:58.435155 21598 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed480872227a4eb586ecf406c78d3ca1" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46115 } } peers { permanent_uuid: "e1a2eb5dd9494f0387bec664455ca098" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 32855 } } peers { permanent_uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 34127 } } }
I20260811 11:57:58.435544 21598 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:57:58.428694 21324 master_runner.cc:186] Error getting master registration for 127.6.143.189:32855: OK, instance_id { permanent_uuid: "e1a2eb5dd9494f0387bec664455ca098" instance_seqno: 1786449478141384 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 11:57:58.438041 21324 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 11:57:58.438174 21400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:57:58.416926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47238 (local address 127.6.143.190:46115)
0811 11:57:58.417028 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 11:57:58.417032 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:57:58.417778 (+ 746us) server_negotiation.cc:407] Connection header received
0811 11:57:58.417956 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:57:58.417959 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:57:58.418004 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:57:58.418057 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:57:58.419031 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.419743 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.420896 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:57:58.421214 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:57:58.421290 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:57:58.423106 (+ 1816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:57:58.423110 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:57:58.423113 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:57:58.423139 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:57:58.424878 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:58.426013 (+ 1135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:58.426017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:58.426019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:58.426103 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:57:58.426434 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:57:58.426437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:57:58.426438 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:57:58.426662 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:57:58.426764 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 11:57:58.427577 (+ 813us) server_negotiation.cc:299] Negotiation successful
0811 11:57:58.428041 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 11:57:58.436028 21489 master_runner.cc:186] Error getting master registration for 127.6.143.189:32855: OK, instance_id { permanent_uuid: "e1a2eb5dd9494f0387bec664455ca098" instance_seqno: 1786449478141384 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 11:57:58.439395 21489 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 11:57:58.440778 21481 raft_consensus.cc:348] T 00000000000000000000000000000000 P e1a2eb5dd9494f0387bec664455ca098 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed480872227a4eb586ecf406c78d3ca1" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46115 } } peers { permanent_uuid: "e1a2eb5dd9494f0387bec664455ca098" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 32855 } } peers { permanent_uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 34127 } }
I20260811 11:57:58.441107 21481 raft_consensus.cc:374] T 00000000000000000000000000000000 P e1a2eb5dd9494f0387bec664455ca098 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:57:58.441190 21481 raft_consensus.cc:729] T 00000000000000000000000000000000 P e1a2eb5dd9494f0387bec664455ca098 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1a2eb5dd9494f0387bec664455ca098, State: Initialized, Role: FOLLOWER
I20260811 11:57:58.441681 21481 consensus_queue.cc:260] T 00000000000000000000000000000000 P e1a2eb5dd9494f0387bec664455ca098 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed480872227a4eb586ecf406c78d3ca1" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46115 } } peers { permanent_uuid: "e1a2eb5dd9494f0387bec664455ca098" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 32855 } } peers { permanent_uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 34127 } }
I20260811 11:57:58.442456 21614 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1a2eb5dd9494f0387bec664455ca098 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed480872227a4eb586ecf406c78d3ca1" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46115 } } peers { permanent_uuid: "e1a2eb5dd9494f0387bec664455ca098" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 32855 } } peers { permanent_uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 34127 } } }
I20260811 11:57:58.442636 21614 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1a2eb5dd9494f0387bec664455ca098 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 11:57:58.443013 21597 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ed480872227a4eb586ecf406c78d3ca1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 11:57:58.443141 21597 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 11:57:58.443130 21481 sys_catalog.cc:564] T 00000000000000000000000000000000 P e1a2eb5dd9494f0387bec664455ca098 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:57:58.453310 21404 master_runner.cc:263] No leader master found from master e1a2eb5dd9494f0387bec664455ca098
I20260811 11:57:58.453380 21404 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 11:57:58.455127 21625 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e1a2eb5dd9494f0387bec664455ca098: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 11:57:58.455267 21625 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 11:57:58.474126 21598 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:57:58.474319 21598 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed480872227a4eb586ecf406c78d3ca1" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46115 } } peers { permanent_uuid: "e1a2eb5dd9494f0387bec664455ca098" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 32855 } } peers { permanent_uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 34127 } }
I20260811 11:57:58.475279 21598 leader_election.cc:288] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed480872227a4eb586ecf406c78d3ca1 (127.6.143.190:46115), e1a2eb5dd9494f0387bec664455ca098 (127.6.143.189:32855)
I20260811 11:57:58.476011 21368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed480872227a4eb586ecf406c78d3ca1" is_pre_election: true
I20260811 11:57:58.476307 21368 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ed480872227a4eb586ecf406c78d3ca1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8812db2a37b49b1b5b4ff5e074b1cbc in term 0.
I20260811 11:57:58.476840 21507 leader_election.cc:302] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8812db2a37b49b1b5b4ff5e074b1cbc, ed480872227a4eb586ecf406c78d3ca1; no voters:
I20260811 11:57:58.477587 21449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1a2eb5dd9494f0387bec664455ca098" is_pre_election: true
I20260811 11:57:58.477874 21449 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e1a2eb5dd9494f0387bec664455ca098 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d8812db2a37b49b1b5b4ff5e074b1cbc in term 0.
I20260811 11:57:58.480309 21598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:57:58.480399 21598 raft_consensus.cc:482] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:57:58.480443 21598 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:57:58.481761 21598 raft_consensus.cc:504] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed480872227a4eb586ecf406c78d3ca1" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46115 } } peers { permanent_uuid: "e1a2eb5dd9494f0387bec664455ca098" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 32855 } } peers { permanent_uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 34127 } }
I20260811 11:57:58.482100 21598 leader_election.cc:288] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc [CANDIDATE]: Term 1 election: Requested vote from peers ed480872227a4eb586ecf406c78d3ca1 (127.6.143.190:46115), e1a2eb5dd9494f0387bec664455ca098 (127.6.143.189:32855)
I20260811 11:57:58.482594 21368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed480872227a4eb586ecf406c78d3ca1"
I20260811 11:57:58.482705 21368 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed480872227a4eb586ecf406c78d3ca1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:57:58.484014 21368 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ed480872227a4eb586ecf406c78d3ca1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8812db2a37b49b1b5b4ff5e074b1cbc in term 1.
I20260811 11:57:58.484377 21507 leader_election.cc:302] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d8812db2a37b49b1b5b4ff5e074b1cbc, ed480872227a4eb586ecf406c78d3ca1; no voters:
I20260811 11:57:58.484902 21449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1a2eb5dd9494f0387bec664455ca098"
I20260811 11:57:58.485018 21449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e1a2eb5dd9494f0387bec664455ca098 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:57:58.486207 21449 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e1a2eb5dd9494f0387bec664455ca098 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d8812db2a37b49b1b5b4ff5e074b1cbc in term 1.
I20260811 11:57:58.486901 21598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:57:58.487203 21598 raft_consensus.cc:686] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc [term 1 LEADER]: Becoming Leader. State: Replica: d8812db2a37b49b1b5b4ff5e074b1cbc, State: Running, Role: LEADER
I20260811 11:57:58.487635 21598 consensus_queue.cc:237] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed480872227a4eb586ecf406c78d3ca1" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46115 } } peers { permanent_uuid: "e1a2eb5dd9494f0387bec664455ca098" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 32855 } } peers { permanent_uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 34127 } }
I20260811 11:57:58.489534 21598 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8812db2a37b49b1b5b4ff5e074b1cbc. Latest consensus state: current_term: 1 leader_uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed480872227a4eb586ecf406c78d3ca1" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46115 } } peers { permanent_uuid: "e1a2eb5dd9494f0387bec664455ca098" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 32855 } } peers { permanent_uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 34127 } } }
I20260811 11:57:58.489697 21598 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc [sys.catalog]: This master's current role is: LEADER
I20260811 11:57:58.490121 21630 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:57:58.491094 21630 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:57:58.500581 21449 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e1a2eb5dd9494f0387bec664455ca098 [term 1 FOLLOWER]: Refusing update from remote peer d8812db2a37b49b1b5b4ff5e074b1cbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:57:58.501310 21598 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1a2eb5dd9494f0387bec664455ca098" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 32855 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:57:58.505545 21614 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1a2eb5dd9494f0387bec664455ca098 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8812db2a37b49b1b5b4ff5e074b1cbc. Latest consensus state: current_term: 1 leader_uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed480872227a4eb586ecf406c78d3ca1" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46115 } } peers { permanent_uuid: "e1a2eb5dd9494f0387bec664455ca098" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 32855 } } peers { permanent_uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 34127 } } }
I20260811 11:57:58.505656 21614 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1a2eb5dd9494f0387bec664455ca098 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:57:58.506688 21368 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ed480872227a4eb586ecf406c78d3ca1 [term 1 FOLLOWER]: Refusing update from remote peer d8812db2a37b49b1b5b4ff5e074b1cbc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:57:58.507290 21598 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed480872227a4eb586ecf406c78d3ca1" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:57:58.511301 21585 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed480872227a4eb586ecf406c78d3ca1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d8812db2a37b49b1b5b4ff5e074b1cbc. Latest consensus state: current_term: 1 leader_uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed480872227a4eb586ecf406c78d3ca1" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46115 } } peers { permanent_uuid: "e1a2eb5dd9494f0387bec664455ca098" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 32855 } } peers { permanent_uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 34127 } } }
I20260811 11:57:58.511427 21585 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed480872227a4eb586ecf406c78d3ca1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:57:58.519520 21628 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed480872227a4eb586ecf406c78d3ca1" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46115 } } peers { permanent_uuid: "e1a2eb5dd9494f0387bec664455ca098" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 32855 } } peers { permanent_uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 34127 } } }
I20260811 11:57:58.519984 21628 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc [sys.catalog]: This master's current role is: LEADER
I20260811 11:57:58.520810 21585 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed480872227a4eb586ecf406c78d3ca1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed480872227a4eb586ecf406c78d3ca1" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46115 } } peers { permanent_uuid: "e1a2eb5dd9494f0387bec664455ca098" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 32855 } } peers { permanent_uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 34127 } } }
I20260811 11:57:58.520924 21585 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed480872227a4eb586ecf406c78d3ca1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:57:58.521472 21631 mvcc.cc:205] Tried to move back new op lower bound from 7317297063931146240 to 7317297063889031168. Current Snapshot: MvccSnapshot[applied={T|T < 7317297063931146240}]
I20260811 11:57:58.521894 21634 mvcc.cc:205] Tried to move back new op lower bound from 7317297063931146240 to 7317297063889031168. Current Snapshot: MvccSnapshot[applied={T|T < 7317297063931146240}]
I20260811 11:57:58.524132 21628 sys_catalog.cc:455] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed480872227a4eb586ecf406c78d3ca1" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46115 } } peers { permanent_uuid: "e1a2eb5dd9494f0387bec664455ca098" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 32855 } } peers { permanent_uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 34127 } } }
I20260811 11:57:58.524231 21628 sys_catalog.cc:458] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc [sys.catalog]: This master's current role is: LEADER
I20260811 11:57:58.525343 21614 sys_catalog.cc:455] T 00000000000000000000000000000000 P e1a2eb5dd9494f0387bec664455ca098 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed480872227a4eb586ecf406c78d3ca1" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46115 } } peers { permanent_uuid: "e1a2eb5dd9494f0387bec664455ca098" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 32855 } } peers { permanent_uuid: "d8812db2a37b49b1b5b4ff5e074b1cbc" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 34127 } } }
I20260811 11:57:58.525455 21614 sys_catalog.cc:458] T 00000000000000000000000000000000 P e1a2eb5dd9494f0387bec664455ca098 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:57:58.529932 21632 mvcc.cc:205] Tried to move back new op lower bound from 7317297063931146240 to 7317297063889031168. Current Snapshot: MvccSnapshot[applied={T|T < 7317297063931146240}]
I20260811 11:57:58.533077 21630 catalog_manager.cc:1347] Generated new cluster ID: 875ecfd5294840938850ac8b6d4ff60b
I20260811 11:57:58.533133 21630 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:57:58.589650 21630 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:57:58.591045 21630 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:57:58.608116 21630 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d8812db2a37b49b1b5b4ff5e074b1cbc: Generated new TSK 0
I20260811 11:57:58.609032 21630 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:57:59.444782 21597 catalog_manager.cc:1259] Loaded cluster ID: 875ecfd5294840938850ac8b6d4ff60b
I20260811 11:57:59.444955 21597 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ed480872227a4eb586ecf406c78d3ca1: loading cluster ID for follower catalog manager: success
I20260811 11:57:59.446875 21597 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ed480872227a4eb586ecf406c78d3ca1: acquiring CA information for follower catalog manager: success
I20260811 11:57:59.448109 21597 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ed480872227a4eb586ecf406c78d3ca1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 11:57:59.456489 21625 catalog_manager.cc:1259] Loaded cluster ID: 875ecfd5294840938850ac8b6d4ff60b
I20260811 11:57:59.456553 21625 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e1a2eb5dd9494f0387bec664455ca098: loading cluster ID for follower catalog manager: success
I20260811 11:57:59.458267 21625 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e1a2eb5dd9494f0387bec664455ca098: acquiring CA information for follower catalog manager: success
I20260811 11:57:59.459431 21625 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e1a2eb5dd9494f0387bec664455ca098: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 11:57:59.678946 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 21324
I20260811 11:57:59.689033 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 21404
I20260811 11:57:59.696587 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 21489
2026-08-11T11:57:59Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3156 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:57:59 dist-test-slave-zkpd krb5kdc[21648](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:57:59 dist-test-slave-zkpd krb5kdc[21648](info): set up 2 sockets
Aug 11 11:57:59 dist-test-slave-zkpd krb5kdc[21648](info): commencing operation
krb5kdc: starting...
W20260811 11:58:00.796882 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:58:00 dist-test-slave-zkpd krb5kdc[21648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:58:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:58:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
I20260811 11:58:00.922405 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:43075
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:33691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:43075
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:58:01.034288 21661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:58:01.034533 21661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:58:01.034572 21661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:58:01.038581 21661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:58:01.038628 21661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:58:01.038647 21661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:58:01.038681 21661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:58:01.038697 21661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:58:01.044355 21661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:33691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:43075
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:43075
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:58:01.045320 21661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:58:01.046331 21661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:58:01.054697 21667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:01.054749 21670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:01.054703 21666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:58:01.055025 21661 server_base.cc:1034] running on GCE node
I20260811 11:58:01.055599 21661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:58:01.056720 21661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:58:01.057906 21661 hybrid_clock.cc:648] HybridClock initialized: now 1786449481057874 us; error 45 us; skew 500 ppm
Aug 11 11:58:01 dist-test-slave-zkpd krb5kdc[21648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449481, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:58:01.065356 21661 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:58:01.066252 21661 webserver.cc:460] Webserver started at http://127.6.143.190:39953/ using document root <none> and password file <none>
I20260811 11:58:01.067035 21661 fs_manager.cc:359] Metadata directory not provided
I20260811 11:58:01.067153 21661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:58:01.067399 21661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:58:01.069523 21661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "656a387f454a479e8c44595260bebef6"
format_stamp: "Formatted at 2026-08-11 11:58:01 on dist-test-slave-zkpd"
server_key: "52f32035f7e469f0d2097a308f764741"
server_key_iv: "3fa0c8187b1df482df9c32c24953ed0d"
server_key_version: "encryptionkey@0"
I20260811 11:58:01.070086 21661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "656a387f454a479e8c44595260bebef6"
format_stamp: "Formatted at 2026-08-11 11:58:01 on dist-test-slave-zkpd"
server_key: "52f32035f7e469f0d2097a308f764741"
server_key_iv: "3fa0c8187b1df482df9c32c24953ed0d"
server_key_version: "encryptionkey@0"
I20260811 11:58:01.073933 21661 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 11:58:01.076287 21678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:01.077461 21661 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:58:01.077597 21661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "656a387f454a479e8c44595260bebef6"
format_stamp: "Formatted at 2026-08-11 11:58:01 on dist-test-slave-zkpd"
server_key: "52f32035f7e469f0d2097a308f764741"
server_key_iv: "3fa0c8187b1df482df9c32c24953ed0d"
server_key_version: "encryptionkey@0"
I20260811 11:58:01.077713 21661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:58:01.090066 21661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:58:01.092682 21661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:58:01.092880 21661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:58:01.100198 21661 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:43075
I20260811 11:58:01.100235 21734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:43075 every 8 connection(s)
I20260811 11:58:01.101086 21661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:58:01.103837 21736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:58:01.108609 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 21661
I20260811 11:58:01.108739 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:58:01.109040 6718 external_mini_cluster.cc:1428] Setting key 78d90a1fddce43daf823501aa55c6d6b
I20260811 11:58:01.109539 21736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 656a387f454a479e8c44595260bebef6: Bootstrap starting.
I20260811 11:58:01.112052 21736 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 656a387f454a479e8c44595260bebef6: Neither blocks nor log segments found. Creating new log.
I20260811 11:58:01.112795 21736 log.cc:824] T 00000000000000000000000000000000 P 656a387f454a479e8c44595260bebef6: Log is configured to *not* fsync() on all Append() calls
I20260811 11:58:01.114646 21736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 656a387f454a479e8c44595260bebef6: No bootstrap required, opened a new log
Aug 11 11:58:01 dist-test-slave-zkpd krb5kdc[21648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449480, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:58:01.118376 21736 raft_consensus.cc:348] T 00000000000000000000000000000000 P 656a387f454a479e8c44595260bebef6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "656a387f454a479e8c44595260bebef6" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 43075 } }
I20260811 11:58:01.118637 21736 raft_consensus.cc:374] T 00000000000000000000000000000000 P 656a387f454a479e8c44595260bebef6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:58:01.118706 21736 raft_consensus.cc:729] T 00000000000000000000000000000000 P 656a387f454a479e8c44595260bebef6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 656a387f454a479e8c44595260bebef6, State: Initialized, Role: FOLLOWER
I20260811 11:58:01.119231 21736 consensus_queue.cc:260] T 00000000000000000000000000000000 P 656a387f454a479e8c44595260bebef6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "656a387f454a479e8c44595260bebef6" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 43075 } }
I20260811 11:58:01.119408 21736 raft_consensus.cc:388] T 00000000000000000000000000000000 P 656a387f454a479e8c44595260bebef6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:58:01.119536 21736 raft_consensus.cc:482] T 00000000000000000000000000000000 P 656a387f454a479e8c44595260bebef6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:58:01.119637 21736 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 656a387f454a479e8c44595260bebef6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:58:01.120541 21736 raft_consensus.cc:504] T 00000000000000000000000000000000 P 656a387f454a479e8c44595260bebef6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "656a387f454a479e8c44595260bebef6" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 43075 } }
I20260811 11:58:01.120893 21736 leader_election.cc:302] T 00000000000000000000000000000000 P 656a387f454a479e8c44595260bebef6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 656a387f454a479e8c44595260bebef6; no voters:
I20260811 11:58:01.121225 21736 leader_election.cc:288] T 00000000000000000000000000000000 P 656a387f454a479e8c44595260bebef6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:58:01.121315 21741 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 656a387f454a479e8c44595260bebef6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:58:01.121495 21741 raft_consensus.cc:686] T 00000000000000000000000000000000 P 656a387f454a479e8c44595260bebef6 [term 1 LEADER]: Becoming Leader. State: Replica: 656a387f454a479e8c44595260bebef6, State: Running, Role: LEADER
I20260811 11:58:01.121785 21741 consensus_queue.cc:237] T 00000000000000000000000000000000 P 656a387f454a479e8c44595260bebef6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "656a387f454a479e8c44595260bebef6" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 43075 } }
I20260811 11:58:01.122490 21736 sys_catalog.cc:564] T 00000000000000000000000000000000 P 656a387f454a479e8c44595260bebef6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:58:01.123508 21743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 656a387f454a479e8c44595260bebef6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "656a387f454a479e8c44595260bebef6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "656a387f454a479e8c44595260bebef6" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 43075 } } }
I20260811 11:58:01.123553 21742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 656a387f454a479e8c44595260bebef6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 656a387f454a479e8c44595260bebef6. Latest consensus state: current_term: 1 leader_uuid: "656a387f454a479e8c44595260bebef6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "656a387f454a479e8c44595260bebef6" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 43075 } } }
I20260811 11:58:01.123636 21743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 656a387f454a479e8c44595260bebef6 [sys.catalog]: This master's current role is: LEADER
I20260811 11:58:01.123661 21742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 656a387f454a479e8c44595260bebef6 [sys.catalog]: This master's current role is: LEADER
I20260811 11:58:01.124163 21750 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:58:01.124191 21739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:01.110365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34428 (local address 127.6.143.190:43075)
0811 11:58:01.110790 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:01.110805 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:01.110840 (+ 35us) server_negotiation.cc:407] Connection header received
0811 11:58:01.111462 (+ 622us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:01.111479 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:01.111796 (+ 317us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:01.112204 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:58:01.113016 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:01.113996 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:01.115078 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:01.115532 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:01.115707 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:01.118230 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:58:01.118235 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:58:01.118247 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:58:01.118274 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:58:01.120287 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:01.120740 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:01.120746 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:01.120752 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:01.120873 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:01.121194 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:01.121197 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:01.121199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:01.121770 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:58:01.122536 (+ 766us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:01.122711 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 11:58:01.123024 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":120,"threads_started":1}
I20260811 11:58:01.127261 6718 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:58:01.128129 21750 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:58:01.134025 21750 catalog_manager.cc:1347] Generated new cluster ID: 121658111e864a96bf04284cf7d52597
I20260811 11:58:01.134080 21750 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:58:01.153609 21750 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:58:01.154548 21750 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:58:01.163146 21750 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 656a387f454a479e8c44595260bebef6: Generated new TSK 0
I20260811 11:58:01.163832 21750 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:58:01.183581 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 21661
2026-08-11T11:58:01Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1487 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:58:01 dist-test-slave-zkpd krb5kdc[21765](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:58:01 dist-test-slave-zkpd krb5kdc[21765](info): set up 2 sockets
Aug 11 11:58:01 dist-test-slave-zkpd krb5kdc[21765](info): commencing operation
krb5kdc: starting...
W20260811 11:58:02.264896 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:58:02 dist-test-slave-zkpd krb5kdc[21765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:58:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:58:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
I20260811 11:58:02.388401 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:39315
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:43795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:39315
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:58:02.498962 21779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:58:02.499212 21779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:58:02.499277 21779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:58:02.503244 21779 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:58:02.503311 21779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:58:02.503333 21779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:58:02.503350 21779 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:58:02.503407 21779 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:58:02.508491 21779 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:43795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:39315
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:39315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:58:02.509438 21779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:58:02.510411 21779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:58:02.518947 21789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:02.518939 21785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:02.518941 21787 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:58:02.519131 21779 server_base.cc:1034] running on GCE node
I20260811 11:58:02.519776 21779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:58:02.520828 21779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:58:02.522022 21779 hybrid_clock.cc:648] HybridClock initialized: now 1786449482521989 us; error 44 us; skew 500 ppm
Aug 11 11:58:02 dist-test-slave-zkpd krb5kdc[21765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449482, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:58:02.524773 21779 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:58:02.525646 21779 webserver.cc:460] Webserver started at http://127.6.143.190:45187/ using document root <none> and password file <none>
I20260811 11:58:02.526196 21779 fs_manager.cc:359] Metadata directory not provided
I20260811 11:58:02.526286 21779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:58:02.526515 21779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:58:02.528546 21779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "6976882a3ff84562a7f6cdf18f466710"
format_stamp: "Formatted at 2026-08-11 11:58:02 on dist-test-slave-zkpd"
server_key: "3575d3614599cad6c8a56e681d140a07"
server_key_iv: "dcedca16a62105884842c8fe10d7e5c5"
server_key_version: "encryptionkey@0"
I20260811 11:58:02.529102 21779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "6976882a3ff84562a7f6cdf18f466710"
format_stamp: "Formatted at 2026-08-11 11:58:02 on dist-test-slave-zkpd"
server_key: "3575d3614599cad6c8a56e681d140a07"
server_key_iv: "dcedca16a62105884842c8fe10d7e5c5"
server_key_version: "encryptionkey@0"
I20260811 11:58:02.532681 21779 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 11:58:02.535043 21799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:02.536295 21779 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:58:02.536433 21779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "6976882a3ff84562a7f6cdf18f466710"
format_stamp: "Formatted at 2026-08-11 11:58:02 on dist-test-slave-zkpd"
server_key: "3575d3614599cad6c8a56e681d140a07"
server_key_iv: "dcedca16a62105884842c8fe10d7e5c5"
server_key_version: "encryptionkey@0"
I20260811 11:58:02.536542 21779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:58:02.555306 21779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:58:02.558522 21779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:58:02.558725 21779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:58:02.566020 21779 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:39315
I20260811 11:58:02.566308 21856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:39315 every 8 connection(s)
I20260811 11:58:02.566991 21779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:58:02.569785 21857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:58:02.574262 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 21779
I20260811 11:58:02.574376 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:58:02.574638 6718 external_mini_cluster.cc:1428] Setting key 1f5ff94b6fb3e0fce28f4442373e202d
I20260811 11:58:02.575963 21857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710: Bootstrap starting.
I20260811 11:58:02.578707 21857 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710: Neither blocks nor log segments found. Creating new log.
I20260811 11:58:02.579435 21857 log.cc:824] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710: Log is configured to *not* fsync() on all Append() calls
I20260811 11:58:02.581302 21857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710: No bootstrap required, opened a new log
Aug 11 11:58:02 dist-test-slave-zkpd krb5kdc[21765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449482, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:58:02.585044 21857 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6976882a3ff84562a7f6cdf18f466710" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39315 } }
I20260811 11:58:02.585285 21857 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:58:02.585371 21857 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6976882a3ff84562a7f6cdf18f466710, State: Initialized, Role: FOLLOWER
I20260811 11:58:02.585800 21857 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6976882a3ff84562a7f6cdf18f466710" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39315 } }
I20260811 11:58:02.585958 21857 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:58:02.586035 21857 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:58:02.586165 21857 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:58:02.586980 21857 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6976882a3ff84562a7f6cdf18f466710" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39315 } }
I20260811 11:58:02.587311 21857 leader_election.cc:302] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6976882a3ff84562a7f6cdf18f466710; no voters:
I20260811 11:58:02.587643 21857 leader_election.cc:288] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:58:02.587713 21862 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:58:02.587992 21862 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [term 1 LEADER]: Becoming Leader. State: Replica: 6976882a3ff84562a7f6cdf18f466710, State: Running, Role: LEADER
I20260811 11:58:02.588987 21857 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:58:02.589013 21860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:02.576107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36404 (local address 127.6.143.190:39315)
0811 11:58:02.576525 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:02.576544 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:02.576586 (+ 42us) server_negotiation.cc:407] Connection header received
0811 11:58:02.577446 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:02.577470 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:02.577868 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:02.578224 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:58:02.578894 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:02.580016 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:02.580843 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:02.581318 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:02.581520 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:02.583937 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:58:02.583944 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:58:02.583966 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:58:02.583993 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:58:02.585874 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:02.586366 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:02.586374 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:02.586384 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:02.586497 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:02.586789 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:02.586792 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:02.586794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:02.587218 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:58:02.587436 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:02.587675 (+ 239us) server_negotiation.cc:299] Negotiation successful
0811 11:58:02.588031 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":98,"threads_started":1}
I20260811 11:58:02.588327 21862 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6976882a3ff84562a7f6cdf18f466710" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39315 } }
I20260811 11:58:02.590797 21864 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6976882a3ff84562a7f6cdf18f466710. Latest consensus state: current_term: 1 leader_uuid: "6976882a3ff84562a7f6cdf18f466710" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6976882a3ff84562a7f6cdf18f466710" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39315 } } }
I20260811 11:58:02.590942 21864 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [sys.catalog]: This master's current role is: LEADER
I20260811 11:58:02.591264 21865 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6976882a3ff84562a7f6cdf18f466710" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6976882a3ff84562a7f6cdf18f466710" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39315 } } }
I20260811 11:58:02.591593 21865 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [sys.catalog]: This master's current role is: LEADER
I20260811 11:58:02.591722 21879 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:58:02.594837 21879 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:58:02.600528 21879 catalog_manager.cc:1347] Generated new cluster ID: 55aabba2cb2c43a28ccb489252d4eb3c
I20260811 11:58:02.600636 21879 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:58:02.607546 21879 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:58:02.608675 21879 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:58:02.620958 21879 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710: Generated new TSK 0
I20260811 11:58:02.621675 21879 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
I20260811 11:58:02.649394 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:0
--local_ip_for_outbound_sockets=127.6.143.129
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:39315
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:43795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:58:02.761315 21886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:58:02.761574 21886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:58:02.761662 21886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:58:02.765646 21886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:58:02.765726 21886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:58:02.765815 21886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:58:02.770780 21886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:43795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:39315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:58:02.771739 21886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:58:02.772769 21886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:58:02.782078 21894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:02.782066 21892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:58:02.782259 21886 server_base.cc:1034] running on GCE node
W20260811 11:58:02.782127 21891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:58:02.782755 21886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:58:02.783349 21886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:58:02.784513 21886 hybrid_clock.cc:648] HybridClock initialized: now 1786449482784499 us; error 27 us; skew 500 ppm
Aug 11 11:58:02 dist-test-slave-zkpd krb5kdc[21765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449482, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:58:02.787248 21886 init.cc:377] Logged in from keytab as kudu/127.6.143.129@KRBTEST.COM (short username kudu)
I20260811 11:58:02.788086 21886 webserver.cc:460] Webserver started at http://127.6.143.129:38909/ using document root <none> and password file <none>
I20260811 11:58:02.788592 21886 fs_manager.cc:359] Metadata directory not provided
I20260811 11:58:02.788652 21886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:58:02.788810 21886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:58:02.791077 21886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/data/instance:
uuid: "e449f197ad66458097ce41adc60d11b0"
format_stamp: "Formatted at 2026-08-11 11:58:02 on dist-test-slave-zkpd"
server_key: "e5cf7993e726d5e81052a82582aac991"
server_key_iv: "70affad25a3f9fa2e6d63c45f5c84c6e"
server_key_version: "encryptionkey@0"
I20260811 11:58:02.791567 21886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance:
uuid: "e449f197ad66458097ce41adc60d11b0"
format_stamp: "Formatted at 2026-08-11 11:58:02 on dist-test-slave-zkpd"
server_key: "e5cf7993e726d5e81052a82582aac991"
server_key_iv: "70affad25a3f9fa2e6d63c45f5c84c6e"
server_key_version: "encryptionkey@0"
I20260811 11:58:02.795167 21886 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 11:58:02.797518 21901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:02.798712 21886 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 11:58:02.798851 21886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "e449f197ad66458097ce41adc60d11b0"
format_stamp: "Formatted at 2026-08-11 11:58:02 on dist-test-slave-zkpd"
server_key: "e5cf7993e726d5e81052a82582aac991"
server_key_iv: "70affad25a3f9fa2e6d63c45f5c84c6e"
server_key_version: "encryptionkey@0"
I20260811 11:58:02.798966 21886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:58:02.808616 21886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:58:02.811379 21886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:58:02.811592 21886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:58:02.812254 21886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:58:02.813117 21886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:58:02.813187 21886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:02.813258 21886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:58:02.813310 21886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:02.822446 21886 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:42901
I20260811 11:58:02.822482 22029 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:42901 every 8 connection(s)
I20260811 11:58:02.823309 21886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:58:02.825832 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 21886
I20260811 11:58:02.825990 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance
I20260811 11:58:02.826313 6718 external_mini_cluster.cc:1428] Setting key cfe553b9cd0cffc23a78820fa880e3bb
Aug 11 11:58:02 dist-test-slave-zkpd krb5kdc[21765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449482, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.143.130@KRBTEST.COM; defaulting to no policy
I20260811 11:58:02.839210 21860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:02.825088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:49909 (local address 127.6.143.190:39315)
0811 11:58:02.825239 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:02.825244 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:02.826263 (+ 1019us) server_negotiation.cc:407] Connection header received
0811 11:58:02.827829 (+ 1566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:02.827832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:02.827912 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:02.828007 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:58:02.829681 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:02.830242 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:02.832053 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:02.832496 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:02.832623 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:02.835306 (+ 2683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:58:02.835313 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:58:02.835318 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:58:02.835352 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:58:02.837120 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:02.837720 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:02.837727 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:02.837730 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:02.837828 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:02.838167 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:02.838170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:02.838172 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:02.838425 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:58:02.838539 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:02.838988 (+ 449us) server_negotiation.cc:299] Negotiation successful
0811 11:58:02.839089 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 11:58:02.840214 22033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:02.825531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.129:49909)
0811 11:58:02.826079 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:02.826139 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:02.827606 (+ 1467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:02.828218 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:02.828227 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:02.828778 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:58:02.829495 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:02.829518 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:02.830380 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:02.830386 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:02.831867 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:02.831879 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:02.832641 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:02.832644 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:02.832786 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:02.833439 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:58:02.833455 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:58:02.835126 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:58:02.837251 (+ 2125us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:02.837258 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:02.837273 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:02.837604 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:02.837938 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:02.837940 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:02.837942 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:02.838071 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:02.838535 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:58:02.838540 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:58:02.838758 (+ 218us) client_negotiation.cc:769] Sending connection context
0811 11:58:02.838918 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 11:58:02.839163 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":109,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.130@KRBTEST.COM" created.
I20260811 11:58:02.841605 22030 heartbeater.cc:347] Connected to a master server at 127.6.143.190:39315
I20260811 11:58:02.841863 22030 heartbeater.cc:464] Registering TS with master...
I20260811 11:58:02.842469 22030 heartbeater.cc:511] Master 127.6.143.190:39315 requested a full tablet report, sending...
I20260811 11:58:02.844339 21820 ts_manager.cc:194] Registered new tserver with Master: e449f197ad66458097ce41adc60d11b0 (127.6.143.129:42901)
I20260811 11:58:02.845528 21820 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.129@KRBTEST.COM'} at 127.6.143.129:49909
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
I20260811 11:58:02.859153 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:0
--local_ip_for_outbound_sockets=127.6.143.130
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:39315
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:43795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:58:02.969638 22036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:58:02.969895 22036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:58:02.969935 22036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:58:02.973757 22036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:58:02.973805 22036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:58:02.973860 22036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:58:02.978755 22036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:43795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:39315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:58:02.979640 22036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:58:02.980631 22036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:58:02.989627 22042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:02.989773 22044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:58:02.989876 22036 server_base.cc:1034] running on GCE node
W20260811 11:58:02.989840 22041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:58:02.990351 22036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:58:02.990984 22036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:58:02.992208 22036 hybrid_clock.cc:648] HybridClock initialized: now 1786449482992182 us; error 48 us; skew 500 ppm
Aug 11 11:58:02 dist-test-slave-zkpd krb5kdc[21765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449482, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:58:02.994839 22036 init.cc:377] Logged in from keytab as kudu/127.6.143.130@KRBTEST.COM (short username kudu)
I20260811 11:58:02.995635 22036 webserver.cc:460] Webserver started at http://127.6.143.130:41815/ using document root <none> and password file <none>
I20260811 11:58:02.996241 22036 fs_manager.cc:359] Metadata directory not provided
I20260811 11:58:02.996333 22036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:58:02.996574 22036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:58:02.998503 22036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/data/instance:
uuid: "2ed0ea19b39a461887cb746015b9d779"
format_stamp: "Formatted at 2026-08-11 11:58:02 on dist-test-slave-zkpd"
server_key: "3708bd5c0a8e756af7ff84b01ce46a26"
server_key_iv: "0fad8a6cffa5babb916453ffa4b9eaba"
server_key_version: "encryptionkey@0"
I20260811 11:58:02.999053 22036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance:
uuid: "2ed0ea19b39a461887cb746015b9d779"
format_stamp: "Formatted at 2026-08-11 11:58:02 on dist-test-slave-zkpd"
server_key: "3708bd5c0a8e756af7ff84b01ce46a26"
server_key_iv: "0fad8a6cffa5babb916453ffa4b9eaba"
server_key_version: "encryptionkey@0"
I20260811 11:58:03.003007 22036 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 11:58:03.005453 22052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:03.006608 22036 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 11:58:03.006742 22036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "2ed0ea19b39a461887cb746015b9d779"
format_stamp: "Formatted at 2026-08-11 11:58:02 on dist-test-slave-zkpd"
server_key: "3708bd5c0a8e756af7ff84b01ce46a26"
server_key_iv: "0fad8a6cffa5babb916453ffa4b9eaba"
server_key_version: "encryptionkey@0"
I20260811 11:58:03.006860 22036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:58:03.030259 22036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:58:03.033051 22036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:58:03.033248 22036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:58:03.033849 22036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:58:03.034735 22036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:58:03.034803 22036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:03.034868 22036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:58:03.034920 22036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:03.043947 22036 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:43291
I20260811 11:58:03.043989 22173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:43291 every 8 connection(s)
I20260811 11:58:03.044819 22036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
I20260811 11:58:03.045677 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 22036
I20260811 11:58:03.045810 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance
I20260811 11:58:03.046108 6718 external_mini_cluster.cc:1428] Setting key 1d22977620a45f40ddd5ae9a36ce400c
Aug 11 11:58:03 dist-test-slave-zkpd krb5kdc[21765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449482, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.143.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.131@KRBTEST.COM" created.
I20260811 11:58:03.059881 21860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:03.048206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:34893 (local address 127.6.143.190:39315)
0811 11:58:03.048336 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:03.048339 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:03.048353 (+ 14us) server_negotiation.cc:407] Connection header received
0811 11:58:03.049913 (+ 1560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:03.049919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:03.049983 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:03.050082 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:58:03.051234 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.052148 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.053370 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.053831 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.053960 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:03.056354 (+ 2394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:58:03.056369 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:58:03.056375 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:58:03.056406 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:58:03.057984 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:03.058475 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:03.058478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:03.058479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:03.058552 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:03.058848 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:03.058850 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:03.058851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:03.059019 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:58:03.059094 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:03.059584 (+ 490us) server_negotiation.cc:299] Negotiation successful
0811 11:58:03.059704 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 11:58:03.060887 22176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:03.047237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.130:34893)
0811 11:58:03.047807 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:03.047866 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:03.049652 (+ 1786us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:03.050271 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:03.050278 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:03.050630 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:58:03.051087 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:03.051100 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.052279 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.052281 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:03.053230 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:03.053240 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.053989 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.053992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:03.054135 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:03.054731 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:58:03.054746 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:58:03.056182 (+ 1436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:58:03.058088 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:03.058092 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:03.058103 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:03.058367 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:03.058635 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:03.058638 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:03.058639 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:03.058768 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:03.059108 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:58:03.059112 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:58:03.059378 (+ 266us) client_negotiation.cc:769] Sending connection context
0811 11:58:03.059543 (+ 165us) client_negotiation.cc:240] Negotiation successful
0811 11:58:03.059824 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":155,"threads_started":1}
I20260811 11:58:03.062037 22174 heartbeater.cc:347] Connected to a master server at 127.6.143.190:39315
I20260811 11:58:03.062296 22174 heartbeater.cc:464] Registering TS with master...
I20260811 11:58:03.062896 22174 heartbeater.cc:511] Master 127.6.143.190:39315 requested a full tablet report, sending...
I20260811 11:58:03.064097 21820 ts_manager.cc:194] Registered new tserver with Master: 2ed0ea19b39a461887cb746015b9d779 (127.6.143.130:43291)
I20260811 11:58:03.064654 21820 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.130@KRBTEST.COM'} at 127.6.143.130:34893
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
I20260811 11:58:03.075361 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:0
--local_ip_for_outbound_sockets=127.6.143.131
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:39315
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:43795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:58:03.188043 22179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:58:03.188256 22179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:58:03.188294 22179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:58:03.192476 22179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:58:03.192521 22179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:58:03.192577 22179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:58:03.197512 22179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:43795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:39315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:58:03.198372 22179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:58:03.199280 22179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:58:03.208873 22184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:03.208825 22187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:03.208827 22185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:58:03.209048 22179 server_base.cc:1034] running on GCE node
I20260811 11:58:03.209563 22179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:58:03.210175 22179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:58:03.211360 22179 hybrid_clock.cc:648] HybridClock initialized: now 1786449483211339 us; error 34 us; skew 500 ppm
Aug 11 11:58:03 dist-test-slave-zkpd krb5kdc[21765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449483, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:58:03.214148 22179 init.cc:377] Logged in from keytab as kudu/127.6.143.131@KRBTEST.COM (short username kudu)
I20260811 11:58:03.214913 22179 webserver.cc:460] Webserver started at http://127.6.143.131:37761/ using document root <none> and password file <none>
I20260811 11:58:03.215428 22179 fs_manager.cc:359] Metadata directory not provided
I20260811 11:58:03.215487 22179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:58:03.215651 22179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:58:03.217713 22179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/data/instance:
uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a"
format_stamp: "Formatted at 2026-08-11 11:58:03 on dist-test-slave-zkpd"
server_key: "216578dd81d975c716227214f1b037d2"
server_key_iv: "2f8b6b94ed337e4fd859c3c296519c55"
server_key_version: "encryptionkey@0"
I20260811 11:58:03.218281 22179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance:
uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a"
format_stamp: "Formatted at 2026-08-11 11:58:03 on dist-test-slave-zkpd"
server_key: "216578dd81d975c716227214f1b037d2"
server_key_iv: "2f8b6b94ed337e4fd859c3c296519c55"
server_key_version: "encryptionkey@0"
I20260811 11:58:03.221876 22179 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 11:58:03.224139 22195 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:03.225301 22179 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:58:03.225430 22179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a"
format_stamp: "Formatted at 2026-08-11 11:58:03 on dist-test-slave-zkpd"
server_key: "216578dd81d975c716227214f1b037d2"
server_key_iv: "2f8b6b94ed337e4fd859c3c296519c55"
server_key_version: "encryptionkey@0"
I20260811 11:58:03.225533 22179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:58:03.240593 22179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:58:03.243691 22179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:58:03.243932 22179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:58:03.244518 22179 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:58:03.245363 22179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:58:03.245411 22179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:03.245477 22179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:58:03.245519 22179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:03.253890 22179 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:41443
I20260811 11:58:03.253954 22319 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:41443 every 8 connection(s)
I20260811 11:58:03.254755 22179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:58:03.261720 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 22179
I20260811 11:58:03.261842 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance
I20260811 11:58:03.262164 6718 external_mini_cluster.cc:1428] Setting key 0b4f52f7abf35fed3c08583edb9a1df8
Aug 11 11:58:03 dist-test-slave-zkpd krb5kdc[21765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449483, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:58:03.267431 21860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:03.256574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:48387 (local address 127.6.143.190:39315)
0811 11:58:03.256717 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:03.256721 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:03.257679 (+ 958us) server_negotiation.cc:407] Connection header received
0811 11:58:03.258538 (+ 859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:03.258540 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:03.258585 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:03.258659 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:58:03.259739 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.260318 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.261396 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.261607 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.261682 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:03.264433 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:58:03.264436 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:58:03.264438 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:58:03.264482 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:58:03.265714 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:03.266177 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:03.266180 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:03.266181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:03.266255 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:03.266546 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:03.266548 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:03.266549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:03.266713 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:58:03.266788 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:03.267219 (+ 431us) server_negotiation.cc:299] Negotiation successful
0811 11:58:03.267314 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 11:58:03.268249 22324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:03.256921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.131:48387)
0811 11:58:03.257557 (+ 636us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:03.257597 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:03.258355 (+ 758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:03.258822 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:03.258829 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:03.259161 (+ 332us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:58:03.259615 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:03.259628 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.260426 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.260429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:03.261287 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:03.261296 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.261766 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.261773 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:03.261979 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:03.262582 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:58:03.262598 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:58:03.264245 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:58:03.265820 (+ 1575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:03.265824 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:03.265836 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:03.266091 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:03.266333 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:03.266335 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:03.266336 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:03.266466 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:03.266799 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:58:03.266804 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:58:03.267023 (+ 219us) client_negotiation.cc:769] Sending connection context
0811 11:58:03.267170 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 11:58:03.267402 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":116,"threads_started":1}
I20260811 11:58:03.269398 22321 heartbeater.cc:347] Connected to a master server at 127.6.143.190:39315
I20260811 11:58:03.269622 22321 heartbeater.cc:464] Registering TS with master...
I20260811 11:58:03.270080 22321 heartbeater.cc:511] Master 127.6.143.190:39315 requested a full tablet report, sending...
I20260811 11:58:03.271065 21820 ts_manager.cc:194] Registered new tserver with Master: fa812b7ff43f40a7b6e2dc0c5fad925a (127.6.143.131:41443)
I20260811 11:58:03.271608 21820 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.131@KRBTEST.COM'} at 127.6.143.131:48387
I20260811 11:58:03.276032 6718 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:58:03.284554 21860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:03.277932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36416 (local address 127.6.143.190:39315)
0811 11:58:03.278057 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:03.278060 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:03.278259 (+ 199us) server_negotiation.cc:407] Connection header received
0811 11:58:03.278357 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:03.278359 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:03.278395 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:03.278468 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:58:03.279077 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.279558 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.280413 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.280615 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.280691 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:03.281887 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:58:03.281890 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:58:03.281892 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:58:03.281913 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:58:03.283110 (+ 1197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:03.283484 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:03.283486 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:03.283487 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:03.283556 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:03.283783 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:03.283785 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:03.283786 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:03.284053 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:58:03.284159 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:03.284356 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 11:58:03.284448 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 11:58:03.287190 21820 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36416:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:58:03.290480 21820 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:58:03.304956 22333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:03.298723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:42901 (local address 127.0.0.1:58644)
0811 11:58:03.299155 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:03.299204 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:03.299341 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:03.300314 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:03.300320 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:03.300349 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:58:03.300497 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:03.300508 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.302095 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.302098 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:03.303323 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:03.303333 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.304417 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.304421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:03.304530 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:03.304571 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 11:58:03.304687 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 11:58:03.304797 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":44,"thread_start_us":85,"threads_started":1}
I20260811 11:58:03.304996 22335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:03.298836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58644 (local address 127.6.143.129:42901)
0811 11:58:03.299685 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:03.299703 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:03.299731 (+ 28us) server_negotiation.cc:407] Connection header received
0811 11:58:03.299818 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:03.299826 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:03.300113 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:03.300681 (+ 568us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:58:03.300853 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.301960 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.303474 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.304289 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.304413 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:03.304568 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:03.304737 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 11:58:03.304853 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":733,"thread_start_us":93,"threads_started":1}
I20260811 11:58:03.305476 22334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:03.298749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:41443 (local address 127.0.0.1:55154)
0811 11:58:03.300199 (+ 1450us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:03.300213 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:03.300314 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:03.301059 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:03.301063 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:03.301083 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:58:03.301191 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:03.301201 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.302376 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.302379 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:03.303780 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:03.303792 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.305068 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.305073 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:03.305168 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:03.305198 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 11:58:03.305269 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 11:58:03.305344 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1382,"mutex_wait_us":83,"thread_start_us":69,"threads_started":1}
I20260811 11:58:03.305804 22337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:03.300239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:43291 (local address 127.0.0.1:41410)
0811 11:58:03.301059 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:03.301072 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:03.301161 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:03.301595 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:03.301598 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:03.301617 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:58:03.301695 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:03.301701 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.303071 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.303074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:03.304123 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:03.304131 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.305379 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.305382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:03.305480 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:03.305499 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 11:58:03.305595 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 11:58:03.305669 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":752,"thread_start_us":89,"threads_started":1}
I20260811 11:58:03.305943 22336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:03.299230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55154 (local address 127.6.143.131:41443)
0811 11:58:03.299708 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:03.299745 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:03.300439 (+ 694us) server_negotiation.cc:407] Connection header received
0811 11:58:03.300526 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:03.300531 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:03.300731 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:03.300882 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:58:03.301292 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.302225 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.303937 (+ 1712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.304752 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.304853 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:03.304996 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:03.305702 (+ 706us) server_negotiation.cc:299] Negotiation successful
0811 11:58:03.305794 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":128,"threads_started":1}
I20260811 11:58:03.306300 22338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:03.300430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41410 (local address 127.6.143.130:43291)
0811 11:58:03.300864 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:03.300876 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:03.301172 (+ 296us) server_negotiation.cc:407] Connection header received
0811 11:58:03.301255 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:03.301260 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:03.301438 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:03.301590 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:58:03.301785 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.302920 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.304245 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.305065 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.305185 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:03.305320 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:03.306053 (+ 733us) server_negotiation.cc:299] Negotiation successful
0811 11:58:03.306151 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":111,"threads_started":1}
I20260811 11:58:03.308718 22246 tablet_service.cc:1467] Processing CreateTablet for tablet e2ffa76507d5436d863c14d07fd7b097 (DEFAULT_TABLE table=test-table [id=cb657bc94ca4475cabc89c2974f43424]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:58:03.308849 21961 tablet_service.cc:1467] Processing CreateTablet for tablet e2ffa76507d5436d863c14d07fd7b097 (DEFAULT_TABLE table=test-table [id=cb657bc94ca4475cabc89c2974f43424]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:58:03.309358 22105 tablet_service.cc:1467] Processing CreateTablet for tablet e2ffa76507d5436d863c14d07fd7b097 (DEFAULT_TABLE table=test-table [id=cb657bc94ca4475cabc89c2974f43424]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:58:03.309819 22246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2ffa76507d5436d863c14d07fd7b097. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:58:03.309998 21961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2ffa76507d5436d863c14d07fd7b097. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:58:03.310402 22105 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e2ffa76507d5436d863c14d07fd7b097. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:58:03.316322 22339 tablet_bootstrap.cc:492] T e2ffa76507d5436d863c14d07fd7b097 P fa812b7ff43f40a7b6e2dc0c5fad925a: Bootstrap starting.
I20260811 11:58:03.318039 22340 tablet_bootstrap.cc:492] T e2ffa76507d5436d863c14d07fd7b097 P e449f197ad66458097ce41adc60d11b0: Bootstrap starting.
I20260811 11:58:03.319644 22339 tablet_bootstrap.cc:654] T e2ffa76507d5436d863c14d07fd7b097 P fa812b7ff43f40a7b6e2dc0c5fad925a: Neither blocks nor log segments found. Creating new log.
I20260811 11:58:03.320212 22341 tablet_bootstrap.cc:492] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779: Bootstrap starting.
I20260811 11:58:03.320502 22340 tablet_bootstrap.cc:654] T e2ffa76507d5436d863c14d07fd7b097 P e449f197ad66458097ce41adc60d11b0: Neither blocks nor log segments found. Creating new log.
I20260811 11:58:03.320896 22339 log.cc:824] T e2ffa76507d5436d863c14d07fd7b097 P fa812b7ff43f40a7b6e2dc0c5fad925a: Log is configured to *not* fsync() on all Append() calls
I20260811 11:58:03.321838 22340 log.cc:824] T e2ffa76507d5436d863c14d07fd7b097 P e449f197ad66458097ce41adc60d11b0: Log is configured to *not* fsync() on all Append() calls
I20260811 11:58:03.322811 22341 tablet_bootstrap.cc:654] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779: Neither blocks nor log segments found. Creating new log.
I20260811 11:58:03.323002 22339 tablet_bootstrap.cc:492] T e2ffa76507d5436d863c14d07fd7b097 P fa812b7ff43f40a7b6e2dc0c5fad925a: No bootstrap required, opened a new log
I20260811 11:58:03.323196 22339 ts_tablet_manager.cc:1397] T e2ffa76507d5436d863c14d07fd7b097 P fa812b7ff43f40a7b6e2dc0c5fad925a: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260811 11:58:03.323606 22341 log.cc:824] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779: Log is configured to *not* fsync() on all Append() calls
I20260811 11:58:03.323861 22340 tablet_bootstrap.cc:492] T e2ffa76507d5436d863c14d07fd7b097 P e449f197ad66458097ce41adc60d11b0: No bootstrap required, opened a new log
I20260811 11:58:03.324116 22340 ts_tablet_manager.cc:1397] T e2ffa76507d5436d863c14d07fd7b097 P e449f197ad66458097ce41adc60d11b0: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260811 11:58:03.325564 22341 tablet_bootstrap.cc:492] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779: No bootstrap required, opened a new log
I20260811 11:58:03.325793 22341 ts_tablet_manager.cc:1397] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260811 11:58:03.326257 22339 raft_consensus.cc:348] T e2ffa76507d5436d863c14d07fd7b097 P fa812b7ff43f40a7b6e2dc0c5fad925a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 } } peers { permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 } } peers { permanent_uuid: "2ed0ea19b39a461887cb746015b9d779" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43291 } }
I20260811 11:58:03.326510 22339 raft_consensus.cc:374] T e2ffa76507d5436d863c14d07fd7b097 P fa812b7ff43f40a7b6e2dc0c5fad925a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:58:03.326565 22339 raft_consensus.cc:729] T e2ffa76507d5436d863c14d07fd7b097 P fa812b7ff43f40a7b6e2dc0c5fad925a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa812b7ff43f40a7b6e2dc0c5fad925a, State: Initialized, Role: FOLLOWER
I20260811 11:58:03.327023 22339 consensus_queue.cc:260] T e2ffa76507d5436d863c14d07fd7b097 P fa812b7ff43f40a7b6e2dc0c5fad925a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 } } peers { permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 } } peers { permanent_uuid: "2ed0ea19b39a461887cb746015b9d779" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43291 } }
I20260811 11:58:03.328085 22339 ts_tablet_manager.cc:1428] T e2ffa76507d5436d863c14d07fd7b097 P fa812b7ff43f40a7b6e2dc0c5fad925a: Time spent starting tablet: real 0.005s user 0.004s sys 0.002s
I20260811 11:58:03.328178 22340 raft_consensus.cc:348] T e2ffa76507d5436d863c14d07fd7b097 P e449f197ad66458097ce41adc60d11b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 } } peers { permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 } } peers { permanent_uuid: "2ed0ea19b39a461887cb746015b9d779" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43291 } }
I20260811 11:58:03.328498 22321 heartbeater.cc:503] Master 127.6.143.190:39315 was elected leader, sending a full tablet report...
I20260811 11:58:03.328512 22340 raft_consensus.cc:374] T e2ffa76507d5436d863c14d07fd7b097 P e449f197ad66458097ce41adc60d11b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:58:03.328578 22340 raft_consensus.cc:729] T e2ffa76507d5436d863c14d07fd7b097 P e449f197ad66458097ce41adc60d11b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e449f197ad66458097ce41adc60d11b0, State: Initialized, Role: FOLLOWER
I20260811 11:58:03.329056 22340 consensus_queue.cc:260] T e2ffa76507d5436d863c14d07fd7b097 P e449f197ad66458097ce41adc60d11b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 } } peers { permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 } } peers { permanent_uuid: "2ed0ea19b39a461887cb746015b9d779" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43291 } }
I20260811 11:58:03.329778 22030 heartbeater.cc:503] Master 127.6.143.190:39315 was elected leader, sending a full tablet report...
I20260811 11:58:03.329843 22341 raft_consensus.cc:348] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 } } peers { permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 } } peers { permanent_uuid: "2ed0ea19b39a461887cb746015b9d779" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43291 } }
I20260811 11:58:03.330106 22341 raft_consensus.cc:374] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:58:03.330235 22341 raft_consensus.cc:729] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ed0ea19b39a461887cb746015b9d779, State: Initialized, Role: FOLLOWER
I20260811 11:58:03.330654 22341 consensus_queue.cc:260] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 } } peers { permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 } } peers { permanent_uuid: "2ed0ea19b39a461887cb746015b9d779" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43291 } }
I20260811 11:58:03.331419 22174 heartbeater.cc:503] Master 127.6.143.190:39315 was elected leader, sending a full tablet report...
I20260811 11:58:03.331751 22341 ts_tablet_manager.cc:1428] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 11:58:03.332058 22340 ts_tablet_manager.cc:1428] T e2ffa76507d5436d863c14d07fd7b097 P e449f197ad66458097ce41adc60d11b0: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260811 11:58:03.443333 22348 raft_consensus.cc:482] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:58:03.443598 22348 raft_consensus.cc:504] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 } } peers { permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 } } peers { permanent_uuid: "2ed0ea19b39a461887cb746015b9d779" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43291 } }
I20260811 11:58:03.444810 22348 leader_election.cc:288] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e449f197ad66458097ce41adc60d11b0 (127.6.143.129:42901), fa812b7ff43f40a7b6e2dc0c5fad925a (127.6.143.131:41443)
I20260811 11:58:03.448926 22335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:03.445428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:36011 (local address 127.6.143.129:42901)
0811 11:58:03.445563 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:03.445566 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:03.445580 (+ 14us) server_negotiation.cc:407] Connection header received
0811 11:58:03.445629 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:03.445631 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:03.445673 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:03.445768 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:58:03.446037 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.446611 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.447975 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.448363 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.448498 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:03.448552 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:03.448733 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 11:58:03.448802 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 11:58:03.448926 22176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:03.445090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:42901 (local address 127.6.143.130:36011)
0811 11:58:03.445201 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:03.445218 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:03.445323 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:03.445785 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:03.445788 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:03.445807 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:58:03.445925 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:03.445931 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.446735 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.446738 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:03.447814 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:03.447824 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.448488 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.448491 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:03.448598 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:03.448627 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 11:58:03.448715 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 11:58:03.448789 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":29}
I20260811 11:58:03.449070 22336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:03.445209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:54293 (local address 127.6.143.131:41443)
0811 11:58:03.445361 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:03.445366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:03.445666 (+ 300us) server_negotiation.cc:407] Connection header received
0811 11:58:03.445830 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:03.445833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:03.445882 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:03.445969 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:58:03.446244 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.446795 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.448093 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.448494 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.448629 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:03.448680 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:03.448868 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 11:58:03.448938 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 11:58:03.449118 22349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:03.445090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:41443 (local address 127.6.143.130:54293)
0811 11:58:03.445553 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:03.445565 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:03.445701 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:03.445983 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:03.445985 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:03.446002 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:58:03.446126 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:03.446132 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.446899 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.446903 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:03.447960 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:03.447972 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.448623 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.448626 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:03.448731 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:03.448752 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 11:58:03.448848 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 11:58:03.448922 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"mutex_wait_us":87,"thread_start_us":127,"threads_started":1}
I20260811 11:58:03.449681 21982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2ffa76507d5436d863c14d07fd7b097" candidate_uuid: "2ed0ea19b39a461887cb746015b9d779" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e449f197ad66458097ce41adc60d11b0" is_pre_election: true
I20260811 11:58:03.449851 22270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2ffa76507d5436d863c14d07fd7b097" candidate_uuid: "2ed0ea19b39a461887cb746015b9d779" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" is_pre_election: true
I20260811 11:58:03.450013 21982 raft_consensus.cc:2454] T e2ffa76507d5436d863c14d07fd7b097 P e449f197ad66458097ce41adc60d11b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ed0ea19b39a461887cb746015b9d779 in term 0.
I20260811 11:58:03.450091 22270 raft_consensus.cc:2454] T e2ffa76507d5436d863c14d07fd7b097 P fa812b7ff43f40a7b6e2dc0c5fad925a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ed0ea19b39a461887cb746015b9d779 in term 0.
I20260811 11:58:03.450567 22053 leader_election.cc:302] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2ed0ea19b39a461887cb746015b9d779, e449f197ad66458097ce41adc60d11b0, fa812b7ff43f40a7b6e2dc0c5fad925a; no voters:
I20260811 11:58:03.450840 22348 raft_consensus.cc:2781] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:58:03.450932 22348 raft_consensus.cc:482] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:58:03.450976 22348 raft_consensus.cc:3037] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:58:03.452015 22348 raft_consensus.cc:504] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 } } peers { permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 } } peers { permanent_uuid: "2ed0ea19b39a461887cb746015b9d779" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43291 } }
I20260811 11:58:03.452320 22348 leader_election.cc:288] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779 [CANDIDATE]: Term 1 election: Requested vote from peers e449f197ad66458097ce41adc60d11b0 (127.6.143.129:42901), fa812b7ff43f40a7b6e2dc0c5fad925a (127.6.143.131:41443)
I20260811 11:58:03.452629 21982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2ffa76507d5436d863c14d07fd7b097" candidate_uuid: "2ed0ea19b39a461887cb746015b9d779" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e449f197ad66458097ce41adc60d11b0"
I20260811 11:58:03.452683 22270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e2ffa76507d5436d863c14d07fd7b097" candidate_uuid: "2ed0ea19b39a461887cb746015b9d779" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a"
I20260811 11:58:03.452754 21982 raft_consensus.cc:3037] T e2ffa76507d5436d863c14d07fd7b097 P e449f197ad66458097ce41adc60d11b0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:58:03.452785 22270 raft_consensus.cc:3037] T e2ffa76507d5436d863c14d07fd7b097 P fa812b7ff43f40a7b6e2dc0c5fad925a [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:58:03.454007 21982 raft_consensus.cc:2454] T e2ffa76507d5436d863c14d07fd7b097 P e449f197ad66458097ce41adc60d11b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ed0ea19b39a461887cb746015b9d779 in term 1.
I20260811 11:58:03.454007 22270 raft_consensus.cc:2454] T e2ffa76507d5436d863c14d07fd7b097 P fa812b7ff43f40a7b6e2dc0c5fad925a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ed0ea19b39a461887cb746015b9d779 in term 1.
I20260811 11:58:03.454278 22053 leader_election.cc:302] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ed0ea19b39a461887cb746015b9d779, fa812b7ff43f40a7b6e2dc0c5fad925a; no voters:
I20260811 11:58:03.454449 22348 raft_consensus.cc:2781] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:58:03.454679 22348 raft_consensus.cc:686] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779 [term 1 LEADER]: Becoming Leader. State: Replica: 2ed0ea19b39a461887cb746015b9d779, State: Running, Role: LEADER
I20260811 11:58:03.455017 22348 consensus_queue.cc:237] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 } } peers { permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 } } peers { permanent_uuid: "2ed0ea19b39a461887cb746015b9d779" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43291 } }
I20260811 11:58:03.458178 21819 catalog_manager.cc:5495] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ed0ea19b39a461887cb746015b9d779 (127.6.143.130). New cstate: current_term: 1 leader_uuid: "2ed0ea19b39a461887cb746015b9d779" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ed0ea19b39a461887cb746015b9d779" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43291 } health_report { overall_health: HEALTHY } } }
I20260811 11:58:03.491961 22338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:03.488440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41424 (local address 127.6.143.130:43291)
0811 11:58:03.488574 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:03.488577 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:03.488589 (+ 12us) server_negotiation.cc:407] Connection header received
0811 11:58:03.488776 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:03.488779 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:03.488836 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:03.488950 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:58:03.489209 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.489735 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.490580 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.490779 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.490882 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:03.491182 (+ 300us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:58:03.491586 (+ 404us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:58:03.491677 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:03.491742 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 11:58:03.491788 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 11:58:03.497779 22270 raft_consensus.cc:1261] T e2ffa76507d5436d863c14d07fd7b097 P fa812b7ff43f40a7b6e2dc0c5fad925a [term 1 FOLLOWER]: Refusing update from remote peer 2ed0ea19b39a461887cb746015b9d779: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:58:03.497778 21982 raft_consensus.cc:1261] T e2ffa76507d5436d863c14d07fd7b097 P e449f197ad66458097ce41adc60d11b0 [term 1 FOLLOWER]: Refusing update from remote peer 2ed0ea19b39a461887cb746015b9d779: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:58:03.498576 22350 consensus_queue.cc:1035] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:58:03.498762 22348 consensus_queue.cc:1035] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 11:58:03.505652 22322 tablet.cc:2369] T e2ffa76507d5436d863c14d07fd7b097 P fa812b7ff43f40a7b6e2dc0c5fad925a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:58:03.507637 22356 mvcc.cc:205] Tried to move back new op lower bound from 7317297084399775744 to 7317297084234489856. Current Snapshot: MvccSnapshot[applied={T|T < 7317297084399775744}]
I20260811 11:58:03.514014 21819 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:58:03.514192 21819 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:58:03.516551 21819 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710: Sending DeleteTablet for 3 replicas of tablet e2ffa76507d5436d863c14d07fd7b097
I20260811 11:58:03.517268 21961 tablet_service.cc:1514] Processing DeleteTablet for tablet e2ffa76507d5436d863c14d07fd7b097 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:58:03 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:58644
I20260811 11:58:03.517375 22105 tablet_service.cc:1514] Processing DeleteTablet for tablet e2ffa76507d5436d863c14d07fd7b097 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:58:03 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:41410
I20260811 11:58:03.517547 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 21886
I20260811 11:58:03.517519 22246 tablet_service.cc:1514] Processing DeleteTablet for tablet e2ffa76507d5436d863c14d07fd7b097 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:58:03 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:55154
I20260811 11:58:03.524130 22367 tablet_replica.cc:330] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779: stopping tablet replica
I20260811 11:58:03.524426 22368 tablet_replica.cc:330] T e2ffa76507d5436d863c14d07fd7b097 P fa812b7ff43f40a7b6e2dc0c5fad925a: stopping tablet replica
I20260811 11:58:03.524439 22367 raft_consensus.cc:2229] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779 [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:58:03.524820 22368 raft_consensus.cc:2229] T e2ffa76507d5436d863c14d07fd7b097 P fa812b7ff43f40a7b6e2dc0c5fad925a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:58:03.524827 22367 raft_consensus.cc:2258] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:58:03.525187 22368 raft_consensus.cc:2258] T e2ffa76507d5436d863c14d07fd7b097 P fa812b7ff43f40a7b6e2dc0c5fad925a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:58:03.526600 22367 ts_tablet_manager.cc:1905] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:58:03.526631 22368 ts_tablet_manager.cc:1905] T e2ffa76507d5436d863c14d07fd7b097 P fa812b7ff43f40a7b6e2dc0c5fad925a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:58:03.529482 22367 ts_tablet_manager.cc:1918] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:58:03.529660 22367 log.cc:1196] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/wal/wals/e2ffa76507d5436d863c14d07fd7b097
W20260811 11:58:03.529749 21804 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.143.129:42901 (error 108)
I20260811 11:58:03.530022 22368 ts_tablet_manager.cc:1918] T e2ffa76507d5436d863c14d07fd7b097 P fa812b7ff43f40a7b6e2dc0c5fad925a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:58:03.530108 22367 ts_tablet_manager.cc:1939] T e2ffa76507d5436d863c14d07fd7b097 P 2ed0ea19b39a461887cb746015b9d779: Deleting consensus metadata
I20260811 11:58:03.530112 22368 log.cc:1196] T e2ffa76507d5436d863c14d07fd7b097 P fa812b7ff43f40a7b6e2dc0c5fad925a: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/wal/wals/e2ffa76507d5436d863c14d07fd7b097
I20260811 11:58:03.530464 22368 ts_tablet_manager.cc:1939] T e2ffa76507d5436d863c14d07fd7b097 P fa812b7ff43f40a7b6e2dc0c5fad925a: Deleting consensus metadata
I20260811 11:58:03.530791 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 22036
I20260811 11:58:03.531831 22337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:03.531521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:42901 (local address 127.0.0.1:58654)
0811 11:58:03.531690 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:03.531751 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.129:42901: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260811 11:58:03.532052 21804 catalog_manager.cc:4584] TS e449f197ad66458097ce41adc60d11b0 (127.6.143.129:42901): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e2ffa76507d5436d863c14d07fd7b097: Network error: Client connection negotiation failed: client connection to 127.6.143.129:42901: connect: Connection refused (error 111)
I20260811 11:58:03.533864 21800 catalog_manager.cc:4841] TS fa812b7ff43f40a7b6e2dc0c5fad925a (127.6.143.131:41443): tablet e2ffa76507d5436d863c14d07fd7b097 (table test-table [id=cb657bc94ca4475cabc89c2974f43424]) successfully deleted
I20260811 11:58:03.539471 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 22179
I20260811 11:58:03.543769 22337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:03.543430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:43291 (local address 127.0.0.1:41430)
0811 11:58:03.543606 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:03.543693 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.130:43291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 11:58:03.548879 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 21779
I20260811 11:58:03.557088 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:39315
--webserver_interface=127.6.143.190
--webserver_port=45187
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:43795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:39315
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:58:03.670258 22371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:58:03.670519 22371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:58:03.670558 22371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:58:03.674532 22371 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:58:03.674582 22371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:58:03.674602 22371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:58:03.674618 22371 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:58:03.674633 22371 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:58:03.679682 22371 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:43795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:39315
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:39315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=45187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:58:03.680670 22371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:58:03.681593 22371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:58:03.695225 22381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:03.695245 22378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:03.695235 22379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:58:03.695492 22371 server_base.cc:1034] running on GCE node
I20260811 11:58:03.696183 22371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:58:03.697271 22371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:58:03.698434 22371 hybrid_clock.cc:648] HybridClock initialized: now 1786449483698419 us; error 26 us; skew 500 ppm
Aug 11 11:58:03 dist-test-slave-zkpd krb5kdc[21765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449483, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:58:03.701314 22371 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:58:03.702176 22371 webserver.cc:460] Webserver started at http://127.6.143.190:45187/ using document root <none> and password file <none>
I20260811 11:58:03.702750 22371 fs_manager.cc:359] Metadata directory not provided
I20260811 11:58:03.702847 22371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:58:03.706643 22371 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260811 11:58:03.708797 22388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:03.709980 22371 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:58:03.710109 22371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "6976882a3ff84562a7f6cdf18f466710"
format_stamp: "Formatted at 2026-08-11 11:58:02 on dist-test-slave-zkpd"
server_key: "3575d3614599cad6c8a56e681d140a07"
server_key_iv: "dcedca16a62105884842c8fe10d7e5c5"
server_key_version: "encryptionkey@0"
I20260811 11:58:03.710683 22371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:58:03.728909 22371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:58:03.732051 22371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:58:03.732262 22371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:58:03.739476 22371 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:39315
I20260811 11:58:03.739758 22446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:39315 every 8 connection(s)
I20260811 11:58:03.740598 22371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:58:03.744031 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 22371
I20260811 11:58:03.744465 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:42901
--local_ip_for_outbound_sockets=127.6.143.129
--tserver_master_addrs=127.6.143.190:39315
--webserver_port=38909
--webserver_interface=127.6.143.129
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:43795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:58:03.745167 22447 sys_catalog.cc:263] Verifying existing consensus state
I20260811 11:58:03.747205 22447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710: Bootstrap starting.
I20260811 11:58:03.772158 22447 log.cc:824] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710: Log is configured to *not* fsync() on all Append() calls
I20260811 11:58:03.783460 22447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 11:58:03.783972 22447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710: Bootstrap complete.
I20260811 11:58:03.789582 22447 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6976882a3ff84562a7f6cdf18f466710" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39315 } }
I20260811 11:58:03.790154 22447 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6976882a3ff84562a7f6cdf18f466710, State: Initialized, Role: FOLLOWER
I20260811 11:58:03.790783 22447 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6976882a3ff84562a7f6cdf18f466710" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39315 } }
I20260811 11:58:03.790951 22447 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:58:03.791033 22447 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:58:03.791131 22447 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [term 1 FOLLOWER]: Advancing to term 2
I20260811 11:58:03.793668 22447 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6976882a3ff84562a7f6cdf18f466710" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39315 } }
I20260811 11:58:03.794076 22447 leader_election.cc:302] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6976882a3ff84562a7f6cdf18f466710; no voters:
I20260811 11:58:03.794440 22447 leader_election.cc:288] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 11:58:03.794574 22453 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 11:58:03.794847 22453 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [term 2 LEADER]: Becoming Leader. State: Replica: 6976882a3ff84562a7f6cdf18f466710, State: Running, Role: LEADER
I20260811 11:58:03.795212 22453 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6976882a3ff84562a7f6cdf18f466710" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39315 } }
I20260811 11:58:03.795698 22447 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:58:03.796548 22456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6976882a3ff84562a7f6cdf18f466710. Latest consensus state: current_term: 2 leader_uuid: "6976882a3ff84562a7f6cdf18f466710" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6976882a3ff84562a7f6cdf18f466710" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39315 } } }
I20260811 11:58:03.796671 22456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [sys.catalog]: This master's current role is: LEADER
I20260811 11:58:03.797344 22455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "6976882a3ff84562a7f6cdf18f466710" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6976882a3ff84562a7f6cdf18f466710" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39315 } } }
I20260811 11:58:03.797469 22455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [sys.catalog]: This master's current role is: LEADER
I20260811 11:58:03.797595 22463 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:58:03.802688 22463 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:58:03.803408 22463 catalog_manager.cc:1259] Loaded cluster ID: 55aabba2cb2c43a28ccb489252d4eb3c
I20260811 11:58:03.803519 22463 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:58:03.805063 22463 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 11:58:03.810088 22463 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 11:58:03.816007 22463 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:58:03.816735 22463 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710: Loaded TSK: 0
I20260811 11:58:03.817930 22463 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 11:58:03.876065 22451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:58:03.876288 22451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:58:03.876328 22451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:58:03.880301 22451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:58:03.880349 22451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:58:03.880437 22451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:58:03.885550 22451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:43795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:42901
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=38909
--tserver_master_addrs=127.6.143.190:39315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:58:03.886518 22451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:58:03.887496 22451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:58:03.896908 22485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:03.896921 22482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:03.896921 22481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:58:03.897506 22451 server_base.cc:1034] running on GCE node
I20260811 11:58:03.897893 22451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:58:03.898488 22451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:58:03.899677 22451 hybrid_clock.cc:648] HybridClock initialized: now 1786449483899652 us; error 41 us; skew 500 ppm
Aug 11 11:58:03 dist-test-slave-zkpd krb5kdc[21765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449483, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:58:03.902524 22451 init.cc:377] Logged in from keytab as kudu/127.6.143.129@KRBTEST.COM (short username kudu)
I20260811 11:58:03.903441 22451 webserver.cc:460] Webserver started at http://127.6.143.129:38909/ using document root <none> and password file <none>
I20260811 11:58:03.904052 22451 fs_manager.cc:359] Metadata directory not provided
I20260811 11:58:03.904158 22451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:58:03.908027 22451 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 11:58:03.910154 22492 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:03.911433 22451 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:58:03.911527 22451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "e449f197ad66458097ce41adc60d11b0"
format_stamp: "Formatted at 2026-08-11 11:58:02 on dist-test-slave-zkpd"
server_key: "e5cf7993e726d5e81052a82582aac991"
server_key_iv: "70affad25a3f9fa2e6d63c45f5c84c6e"
server_key_version: "encryptionkey@0"
I20260811 11:58:03.912189 22451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:58:03.936887 22451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:58:03.939992 22451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:58:03.940191 22451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:58:03.940865 22451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:58:03.941993 22500 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 11:58:03.946194 22451 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 11:58:03.946257 22451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.001s sys 0.000s
I20260811 11:58:03.946305 22451 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 11:58:03.947949 22451 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 11:58:03.948007 22451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260811 11:58:03.948273 22500 tablet_bootstrap.cc:492] T e2ffa76507d5436d863c14d07fd7b097 P e449f197ad66458097ce41adc60d11b0: Bootstrap starting.
I20260811 11:58:03.959219 22451 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:42901
I20260811 11:58:03.959239 22618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:42901 every 8 connection(s)
I20260811 11:58:03.960502 22451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:58:03.963093 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 22451
I20260811 11:58:03.963603 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:43291
--local_ip_for_outbound_sockets=127.6.143.130
--tserver_master_addrs=127.6.143.190:39315
--webserver_port=41815
--webserver_interface=127.6.143.130
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:43795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:58:03.969622 22500 log.cc:824] T e2ffa76507d5436d863c14d07fd7b097 P e449f197ad66458097ce41adc60d11b0: Log is configured to *not* fsync() on all Append() calls
Aug 11 11:58:03 dist-test-slave-zkpd krb5kdc[21765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449483, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:58:03.977571 22500 tablet_bootstrap.cc:492] T e2ffa76507d5436d863c14d07fd7b097 P e449f197ad66458097ce41adc60d11b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 11:58:03.978030 22500 tablet_bootstrap.cc:492] T e2ffa76507d5436d863c14d07fd7b097 P e449f197ad66458097ce41adc60d11b0: Bootstrap complete.
I20260811 11:58:03.979388 22500 ts_tablet_manager.cc:1397] T e2ffa76507d5436d863c14d07fd7b097 P e449f197ad66458097ce41adc60d11b0: Time spent bootstrapping tablet: real 0.031s user 0.019s sys 0.009s
I20260811 11:58:03.981199 22622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:03.962719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.129:58655)
0811 11:58:03.963659 (+ 940us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:03.963719 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:03.964982 (+ 1263us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:03.966511 (+ 1529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:03.966522 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:03.967031 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:58:03.967556 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:03.967571 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.969417 (+ 1846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.969424 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:03.970985 (+ 1561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:03.971002 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.972003 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.972007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:03.972241 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:03.972997 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:58:03.973023 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:58:03.975019 (+ 1996us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:58:03.977835 (+ 2816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:03.977845 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:03.977860 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:03.978193 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:03.978570 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:03.978576 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:03.978580 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:03.978759 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:03.979503 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:58:03.979510 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:58:03.979785 (+ 275us) client_negotiation.cc:769] Sending connection context
0811 11:58:03.980016 (+ 231us) client_negotiation.cc:240] Negotiation successful
0811 11:58:03.980324 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"thread_start_us":172,"threads_started":1}
I20260811 11:58:03.981230 22621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:03.962706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:58655 (local address 127.6.143.190:39315)
0811 11:58:03.963267 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:03.963287 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:03.963879 (+ 592us) server_negotiation.cc:407] Connection header received
0811 11:58:03.965479 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:03.965495 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:03.965992 (+ 497us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:03.966423 (+ 431us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:58:03.967742 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.969223 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.971144 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.971827 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.972055 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:03.975241 (+ 3186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:58:03.975251 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:58:03.975272 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:58:03.975312 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:58:03.977678 (+ 2366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:03.978305 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:03.978313 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:03.978319 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:03.978458 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:03.978856 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:03.978860 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:03.978861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:03.979300 (+ 439us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:58:03.979434 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:03.980159 (+ 725us) server_negotiation.cc:299] Negotiation successful
0811 11:58:03.980458 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":171,"threads_started":1}
I20260811 11:58:03.982075 22500 raft_consensus.cc:348] T e2ffa76507d5436d863c14d07fd7b097 P e449f197ad66458097ce41adc60d11b0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 } } peers { permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 } } peers { permanent_uuid: "2ed0ea19b39a461887cb746015b9d779" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43291 } }
I20260811 11:58:03.982523 22500 raft_consensus.cc:729] T e2ffa76507d5436d863c14d07fd7b097 P e449f197ad66458097ce41adc60d11b0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e449f197ad66458097ce41adc60d11b0, State: Initialized, Role: FOLLOWER
I20260811 11:58:03.983014 22500 consensus_queue.cc:260] T e2ffa76507d5436d863c14d07fd7b097 P e449f197ad66458097ce41adc60d11b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 } } peers { permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 } } peers { permanent_uuid: "2ed0ea19b39a461887cb746015b9d779" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43291 } }
I20260811 11:58:03.983953 22500 ts_tablet_manager.cc:1428] T e2ffa76507d5436d863c14d07fd7b097 P e449f197ad66458097ce41adc60d11b0: Time spent starting tablet: real 0.004s user 0.008s sys 0.000s
I20260811 11:58:03.984344 22619 heartbeater.cc:347] Connected to a master server at 127.6.143.190:39315
I20260811 11:58:03.984558 22619 heartbeater.cc:464] Registering TS with master...
I20260811 11:58:03.985057 22619 heartbeater.cc:511] Master 127.6.143.190:39315 requested a full tablet report, sending...
I20260811 11:58:03.987443 22406 ts_manager.cc:194] Registered new tserver with Master: e449f197ad66458097ce41adc60d11b0 (127.6.143.129:42901)
I20260811 11:58:03.990931 22406 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.129@KRBTEST.COM'} at 127.6.143.129:58655
I20260811 11:58:03.993470 22619 heartbeater.cc:503] Master 127.6.143.190:39315 was elected leader, sending a full tablet report...
Aug 11 11:58:03 dist-test-slave-zkpd krb5kdc[21765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449483, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for kudu/127.6.143.129@KRBTEST.COM
I20260811 11:58:04.001206 22628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:03.990401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58668 (local address 127.6.143.129:42901)
0811 11:58:03.990724 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:03.990736 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:03.991113 (+ 377us) server_negotiation.cc:407] Connection header received
0811 11:58:03.991375 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:03.991381 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:03.991565 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:03.991698 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:58:03.992492 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.993488 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.994547 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.994924 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.995023 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:03.997485 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:58:03.997495 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:58:03.997507 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:58:03.997533 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:58:03.999275 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:03.999743 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:03.999749 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:03.999751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:03.999843 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:04.000196 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:04.000200 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:04.000202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:04.000444 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:58:04.000592 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:04.000918 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 11:58:04.001070 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":90,"threads_started":1}
I20260811 11:58:04.001201 22627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:03.990303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:42901 (local address 127.0.0.1:58668)
0811 11:58:03.990991 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:03.991034 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:03.991208 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:03.991820 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:03.991825 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:03.992245 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:58:03.992362 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:03.992370 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.993632 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.993635 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:03.994420 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:03.994429 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:03.995153 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:03.995157 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:03.995265 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:03.995864 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:58:03.995882 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:58:03.997349 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:58:03.999380 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:03.999388 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:03.999392 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:03.999601 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:03.999984 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:03.999987 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:03.999991 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:04.000073 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:04.000583 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:58:04.000589 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:58:04.000700 (+ 111us) client_negotiation.cc:769] Sending connection context
0811 11:58:04.000837 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 11:58:04.001029 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":92,"threads_started":1}
I20260811 11:58:04.002271 22547 tablet_service.cc:1514] Processing DeleteTablet for tablet e2ffa76507d5436d863c14d07fd7b097 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:58:03 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:58668
I20260811 11:58:04.003098 22629 tablet_replica.cc:330] stopping tablet replica
I20260811 11:58:04.003388 22629 raft_consensus.cc:2229] T e2ffa76507d5436d863c14d07fd7b097 P e449f197ad66458097ce41adc60d11b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:58:04.003633 22629 raft_consensus.cc:2258] T e2ffa76507d5436d863c14d07fd7b097 P e449f197ad66458097ce41adc60d11b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:58:04.004667 22629 ts_tablet_manager.cc:1905] T e2ffa76507d5436d863c14d07fd7b097 P e449f197ad66458097ce41adc60d11b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:58:04.008268 22629 ts_tablet_manager.cc:1918] T e2ffa76507d5436d863c14d07fd7b097 P e449f197ad66458097ce41adc60d11b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:58:04.008354 22629 log.cc:1196] T e2ffa76507d5436d863c14d07fd7b097 P e449f197ad66458097ce41adc60d11b0: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/wal/wals/e2ffa76507d5436d863c14d07fd7b097
I20260811 11:58:04.008669 22629 ts_tablet_manager.cc:1939] T e2ffa76507d5436d863c14d07fd7b097 P e449f197ad66458097ce41adc60d11b0: Deleting consensus metadata
I20260811 11:58:04.010326 22392 catalog_manager.cc:4841] TS e449f197ad66458097ce41adc60d11b0 (127.6.143.129:42901): tablet e2ffa76507d5436d863c14d07fd7b097 (table test-table [id=cb657bc94ca4475cabc89c2974f43424]) successfully deleted
W20260811 11:58:04.084790 22624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:58:04.085052 22624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:58:04.085124 22624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:58:04.089022 22624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:58:04.089099 22624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:58:04.089181 22624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:58:04.094547 22624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:43795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:43291
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=41815
--tserver_master_addrs=127.6.143.190:39315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:58:04.095513 22624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:58:04.096614 22624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:58:04.106480 22635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:04.106490 22637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:04.106472 22634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:58:04.106772 22624 server_base.cc:1034] running on GCE node
I20260811 11:58:04.107221 22624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:58:04.107821 22624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:58:04.108982 22624 hybrid_clock.cc:648] HybridClock initialized: now 1786449484108963 us; error 29 us; skew 500 ppm
Aug 11 11:58:04 dist-test-slave-zkpd krb5kdc[21765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449484, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:58:04.111716 22624 init.cc:377] Logged in from keytab as kudu/127.6.143.130@KRBTEST.COM (short username kudu)
I20260811 11:58:04.112629 22624 webserver.cc:460] Webserver started at http://127.6.143.130:41815/ using document root <none> and password file <none>
I20260811 11:58:04.113133 22624 fs_manager.cc:359] Metadata directory not provided
I20260811 11:58:04.113202 22624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:58:04.117273 22624 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 11:58:04.119207 22645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:04.120498 22624 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:58:04.120633 22624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "2ed0ea19b39a461887cb746015b9d779"
format_stamp: "Formatted at 2026-08-11 11:58:02 on dist-test-slave-zkpd"
server_key: "3708bd5c0a8e756af7ff84b01ce46a26"
server_key_iv: "0fad8a6cffa5babb916453ffa4b9eaba"
server_key_version: "encryptionkey@0"
I20260811 11:58:04.121168 22624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:58:04.148860 22624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:58:04.151718 22624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:58:04.151999 22624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:58:04.152601 22624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:58:04.153452 22624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:58:04.153524 22624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:04.153594 22624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:58:04.153640 22624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:04.162742 22624 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:43291
I20260811 11:58:04.162778 22770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:43291 every 8 connection(s)
I20260811 11:58:04.163719 22624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
I20260811 11:58:04.172435 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 22624
I20260811 11:58:04.172912 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:41443
--local_ip_for_outbound_sockets=127.6.143.131
--tserver_master_addrs=127.6.143.190:39315
--webserver_port=37761
--webserver_interface=127.6.143.131
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:43795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 11:58:04 dist-test-slave-zkpd krb5kdc[21765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449484, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:58:04.177855 22621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:04.165531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:34099 (local address 127.6.143.190:39315)
0811 11:58:04.165675 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:04.165678 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:04.166431 (+ 753us) server_negotiation.cc:407] Connection header received
0811 11:58:04.167726 (+ 1295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:04.167730 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:04.167801 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:04.167925 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:58:04.169117 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:04.169650 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:04.170787 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:04.171033 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:04.171149 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:04.173842 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:58:04.173849 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:58:04.173862 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:58:04.173899 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:58:04.175438 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:04.176101 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:04.176104 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:04.176105 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:04.176227 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:04.176682 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:04.176685 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:04.176686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:04.176862 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:58:04.176977 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:04.177624 (+ 647us) server_negotiation.cc:299] Negotiation successful
0811 11:58:04.177739 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 11:58:04.179051 22773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:04.165832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.130:34099)
0811 11:58:04.166281 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:04.166322 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:04.167507 (+ 1185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:04.168123 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:04.168131 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:04.168489 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:58:04.168959 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:04.168973 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:04.169785 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:04.169789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:04.170654 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:04.170663 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:04.171187 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:04.171191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:04.171362 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:04.171968 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:58:04.171983 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:58:04.173654 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:58:04.175563 (+ 1909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:04.175572 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:04.175588 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:04.176001 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:04.176366 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:04.176369 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:04.176372 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:04.176563 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:04.176990 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:58:04.176999 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:58:04.177349 (+ 350us) client_negotiation.cc:769] Sending connection context
0811 11:58:04.177564 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 11:58:04.177912 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":105,"threads_started":1}
I20260811 11:58:04.180269 22771 heartbeater.cc:347] Connected to a master server at 127.6.143.190:39315
I20260811 11:58:04.180552 22771 heartbeater.cc:464] Registering TS with master...
I20260811 11:58:04.181186 22771 heartbeater.cc:511] Master 127.6.143.190:39315 requested a full tablet report, sending...
I20260811 11:58:04.182330 22406 ts_manager.cc:194] Registered new tserver with Master: 2ed0ea19b39a461887cb746015b9d779 (127.6.143.130:43291)
I20260811 11:58:04.182890 22406 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.130@KRBTEST.COM'} at 127.6.143.130:34099
W20260811 11:58:04.291918 22774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:58:04.292275 22774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:58:04.292337 22774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:58:04.296512 22774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:58:04.296567 22774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:58:04.296624 22774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:58:04.302110 22774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:43795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:41443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=37761
--tserver_master_addrs=127.6.143.190:39315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:58:04.303279 22774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:58:04.304350 22774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:58:04.313830 22781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:04.313827 22783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:58:04.313927 22774 server_base.cc:1034] running on GCE node
W20260811 11:58:04.313805 22780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:58:04.314518 22774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:58:04.315155 22774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:58:04.316344 22774 hybrid_clock.cc:648] HybridClock initialized: now 1786449484316303 us; error 52 us; skew 500 ppm
Aug 11 11:58:04 dist-test-slave-zkpd krb5kdc[21765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449484, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:58:04.319147 22774 init.cc:377] Logged in from keytab as kudu/127.6.143.131@KRBTEST.COM (short username kudu)
I20260811 11:58:04.320129 22774 webserver.cc:460] Webserver started at http://127.6.143.131:37761/ using document root <none> and password file <none>
I20260811 11:58:04.320704 22774 fs_manager.cc:359] Metadata directory not provided
I20260811 11:58:04.320825 22774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:58:04.325639 22774 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 11:58:04.327688 22790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:04.328925 22774 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:58:04.329066 22774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a"
format_stamp: "Formatted at 2026-08-11 11:58:03 on dist-test-slave-zkpd"
server_key: "216578dd81d975c716227214f1b037d2"
server_key_iv: "2f8b6b94ed337e4fd859c3c296519c55"
server_key_version: "encryptionkey@0"
I20260811 11:58:04.329608 22774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:58:04.343093 22774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:58:04.345837 22774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:58:04.346031 22774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:58:04.346597 22774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:58:04.347425 22774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:58:04.347496 22774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:04.347568 22774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:58:04.347626 22774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:04.356557 22774 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:41443
I20260811 11:58:04.356580 22914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:41443 every 8 connection(s)
I20260811 11:58:04.357518 22774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:58:04.361217 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 22774
Aug 11 11:58:04 dist-test-slave-zkpd krb5kdc[21765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449484, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:58:04.370702 22919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:04.361930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36424 (local address 127.6.143.190:39315)
0811 11:58:04.362180 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:04.362185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:04.362320 (+ 135us) server_negotiation.cc:407] Connection header received
0811 11:58:04.362502 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:04.362505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:04.362559 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:04.362665 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:58:04.363473 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:04.364136 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:04.365106 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:04.365364 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:04.365486 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:04.367076 (+ 1590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:58:04.367080 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:58:04.367082 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:58:04.367107 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:58:04.368820 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:04.369307 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:04.369316 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:04.369317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:04.369403 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:04.369717 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:04.369722 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:04.369724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:04.369940 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:58:04.370168 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:04.370366 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 11:58:04.370503 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":76,"threads_started":1}
I20260811 11:58:04.371647 22621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:04.359398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:37619 (local address 127.6.143.190:39315)
0811 11:58:04.359561 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:04.359565 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:04.360390 (+ 825us) server_negotiation.cc:407] Connection header received
0811 11:58:04.361875 (+ 1485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:04.361878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:04.361954 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:04.362050 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:58:04.363281 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:04.363794 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:04.364956 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:04.365175 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:04.365285 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:04.367949 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:58:04.367952 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:58:04.367954 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:58:04.368013 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:58:04.369467 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:04.370036 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:04.370040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:04.370042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:04.370123 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:04.370475 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:04.370478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:04.370480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:04.370707 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:58:04.370821 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:04.371389 (+ 568us) server_negotiation.cc:299] Negotiation successful
0811 11:58:04.371514 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 11:58:04.372707 22917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:04.359743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.131:37619)
0811 11:58:04.360223 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:04.360269 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:04.361611 (+ 1342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:04.362311 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:04.362318 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:04.362667 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:58:04.363135 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:04.363148 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:04.363955 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:04.363959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:04.364819 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:04.364829 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:04.365328 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:04.365332 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:04.365505 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:04.366124 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:58:04.366149 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:58:04.367742 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:58:04.369593 (+ 1851us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:04.369599 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:04.369614 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:04.369925 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:04.370217 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:04.370221 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:04.370223 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:04.370368 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:04.370822 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:58:04.370829 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:58:04.371126 (+ 297us) client_negotiation.cc:769] Sending connection context
0811 11:58:04.371342 (+ 216us) client_negotiation.cc:240] Negotiation successful
0811 11:58:04.371658 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":135,"threads_started":1}
I20260811 11:58:04.373829 22915 heartbeater.cc:347] Connected to a master server at 127.6.143.190:39315
I20260811 11:58:04.374047 22915 heartbeater.cc:464] Registering TS with master...
I20260811 11:58:04.374492 22915 heartbeater.cc:511] Master 127.6.143.190:39315 requested a full tablet report, sending...
I20260811 11:58:04.375473 22406 ts_manager.cc:194] Registered new tserver with Master: fa812b7ff43f40a7b6e2dc0c5fad925a (127.6.143.131:41443)
I20260811 11:58:04.376174 22406 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.131@KRBTEST.COM'} at 127.6.143.131:37619
I20260811 11:58:04.384545 6718 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:58:04.388408 22621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:04.385135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36426 (local address 127.6.143.190:39315)
0811 11:58:04.385251 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:04.385255 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:04.385412 (+ 157us) server_negotiation.cc:407] Connection header received
0811 11:58:04.385518 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:04.385521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:04.385610 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:04.385695 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:58:04.385876 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:04.386346 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:04.387127 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:04.387345 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:04.387429 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:04.387693 (+ 264us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:58:04.388115 (+ 422us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:58:04.388204 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:04.388269 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 11:58:04.388311 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 11:58:04.389002 22406 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:36426:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:58:04.392257 22406 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:58:04.399736 22547 tablet_service.cc:1467] Processing CreateTablet for tablet 1ccbdfc56ca64bb2b2bb91d14877347b (DEFAULT_TABLE table=test-table [id=9a5f1bab55144315b3baa3fc3da50e51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:58:04.400490 22547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ccbdfc56ca64bb2b2bb91d14877347b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:58:04.403543 22627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:04.398555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:43291 (local address 127.0.0.1:41432)
0811 11:58:04.398653 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:04.398667 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:04.398745 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:04.399680 (+ 935us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:04.399683 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:04.399704 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:58:04.399802 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:04.399809 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:04.400997 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:04.401003 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:04.401775 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:04.401784 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:04.403129 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:04.403132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:04.403226 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:04.403265 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 11:58:04.403343 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 11:58:04.403426 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260811 11:58:04.403605 22924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:04.398729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41432 (local address 127.6.143.130:43291)
0811 11:58:04.399100 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:04.399112 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:04.399136 (+ 24us) server_negotiation.cc:407] Connection header received
0811 11:58:04.399214 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:04.399253 (+ 39us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:04.399495 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:04.399691 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:58:04.399937 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:04.400860 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:04.401914 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:04.403001 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:04.403128 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:04.403275 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:04.403383 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 11:58:04.403472 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":133,"threads_started":1}
I20260811 11:58:04.403771 22500 tablet_bootstrap.cc:492] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0: Bootstrap starting.
I20260811 11:58:04.404219 22925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:04.398777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:41443 (local address 127.0.0.1:55162)
0811 11:58:04.399361 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:04.399375 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:04.399460 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:04.400197 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:04.400200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:04.400219 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:58:04.400299 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:04.400305 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:04.401618 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:04.401622 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:04.402696 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:04.402707 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:04.403804 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:04.403807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:04.403926 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:04.403949 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 11:58:04.404023 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 11:58:04.404097 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":97,"threads_started":1}
I20260811 11:58:04.404266 22926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:04.399206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55162 (local address 127.6.143.131:41443)
0811 11:58:04.399713 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:04.399724 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:04.399749 (+ 25us) server_negotiation.cc:407] Connection header received
0811 11:58:04.399819 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:04.399826 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:04.400046 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:04.400203 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:58:04.400436 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:04.401458 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:04.402841 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:04.403654 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:04.403797 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:04.403953 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:04.404059 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 11:58:04.404138 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":159,"threads_started":1}
I20260811 11:58:04.404582 22500 tablet_bootstrap.cc:654] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0: Neither blocks nor log segments found. Creating new log.
I20260811 11:58:04.406009 22500 tablet_bootstrap.cc:492] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0: No bootstrap required, opened a new log
I20260811 11:58:04.406077 22500 ts_tablet_manager.cc:1397] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 11:58:04.406531 22500 raft_consensus.cc:348] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed0ea19b39a461887cb746015b9d779" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43291 } } peers { permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 } } peers { permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 } }
I20260811 11:58:04.406652 22500 raft_consensus.cc:374] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:58:04.406690 22500 raft_consensus.cc:729] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e449f197ad66458097ce41adc60d11b0, State: Initialized, Role: FOLLOWER
I20260811 11:58:04.406813 22500 consensus_queue.cc:260] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed0ea19b39a461887cb746015b9d779" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43291 } } peers { permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 } } peers { permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 } }
I20260811 11:58:04.407200 22500 ts_tablet_manager.cc:1428] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 11:58:04.408797 22841 tablet_service.cc:1467] Processing CreateTablet for tablet 1ccbdfc56ca64bb2b2bb91d14877347b (DEFAULT_TABLE table=test-table [id=9a5f1bab55144315b3baa3fc3da50e51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:58:04.408753 22697 tablet_service.cc:1467] Processing CreateTablet for tablet 1ccbdfc56ca64bb2b2bb91d14877347b (DEFAULT_TABLE table=test-table [id=9a5f1bab55144315b3baa3fc3da50e51]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:58:04.409873 22697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ccbdfc56ca64bb2b2bb91d14877347b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:58:04.410181 22841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1ccbdfc56ca64bb2b2bb91d14877347b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:58:04.416733 22928 tablet_bootstrap.cc:492] T 1ccbdfc56ca64bb2b2bb91d14877347b P fa812b7ff43f40a7b6e2dc0c5fad925a: Bootstrap starting.
I20260811 11:58:04.416790 22930 tablet_bootstrap.cc:492] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779: Bootstrap starting.
I20260811 11:58:04.419242 22930 tablet_bootstrap.cc:654] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779: Neither blocks nor log segments found. Creating new log.
I20260811 11:58:04.419322 22928 tablet_bootstrap.cc:654] T 1ccbdfc56ca64bb2b2bb91d14877347b P fa812b7ff43f40a7b6e2dc0c5fad925a: Neither blocks nor log segments found. Creating new log.
I20260811 11:58:04.420035 22928 log.cc:824] T 1ccbdfc56ca64bb2b2bb91d14877347b P fa812b7ff43f40a7b6e2dc0c5fad925a: Log is configured to *not* fsync() on all Append() calls
I20260811 11:58:04.420030 22930 log.cc:824] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779: Log is configured to *not* fsync() on all Append() calls
I20260811 11:58:04.421758 22930 tablet_bootstrap.cc:492] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779: No bootstrap required, opened a new log
I20260811 11:58:04.421767 22928 tablet_bootstrap.cc:492] T 1ccbdfc56ca64bb2b2bb91d14877347b P fa812b7ff43f40a7b6e2dc0c5fad925a: No bootstrap required, opened a new log
I20260811 11:58:04.421945 22930 ts_tablet_manager.cc:1397] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 11:58:04.421962 22928 ts_tablet_manager.cc:1397] T 1ccbdfc56ca64bb2b2bb91d14877347b P fa812b7ff43f40a7b6e2dc0c5fad925a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 11:58:04.424985 22928 raft_consensus.cc:348] T 1ccbdfc56ca64bb2b2bb91d14877347b P fa812b7ff43f40a7b6e2dc0c5fad925a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed0ea19b39a461887cb746015b9d779" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43291 } } peers { permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 } } peers { permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 } }
I20260811 11:58:04.424985 22930 raft_consensus.cc:348] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed0ea19b39a461887cb746015b9d779" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43291 } } peers { permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 } } peers { permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 } }
I20260811 11:58:04.425225 22928 raft_consensus.cc:374] T 1ccbdfc56ca64bb2b2bb91d14877347b P fa812b7ff43f40a7b6e2dc0c5fad925a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:58:04.425246 22930 raft_consensus.cc:374] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:58:04.425298 22930 raft_consensus.cc:729] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ed0ea19b39a461887cb746015b9d779, State: Initialized, Role: FOLLOWER
I20260811 11:58:04.425300 22928 raft_consensus.cc:729] T 1ccbdfc56ca64bb2b2bb91d14877347b P fa812b7ff43f40a7b6e2dc0c5fad925a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa812b7ff43f40a7b6e2dc0c5fad925a, State: Initialized, Role: FOLLOWER
I20260811 11:58:04.425719 22930 consensus_queue.cc:260] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed0ea19b39a461887cb746015b9d779" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43291 } } peers { permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 } } peers { permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 } }
I20260811 11:58:04.425719 22928 consensus_queue.cc:260] T 1ccbdfc56ca64bb2b2bb91d14877347b P fa812b7ff43f40a7b6e2dc0c5fad925a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed0ea19b39a461887cb746015b9d779" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43291 } } peers { permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 } } peers { permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 } }
I20260811 11:58:04.426328 22771 heartbeater.cc:503] Master 127.6.143.190:39315 was elected leader, sending a full tablet report...
I20260811 11:58:04.426328 22915 heartbeater.cc:503] Master 127.6.143.190:39315 was elected leader, sending a full tablet report...
I20260811 11:58:04.426671 22928 ts_tablet_manager.cc:1428] T 1ccbdfc56ca64bb2b2bb91d14877347b P fa812b7ff43f40a7b6e2dc0c5fad925a: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 11:58:04.426671 22930 ts_tablet_manager.cc:1428] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260811 11:58:04.461689 22620 tablet.cc:2369] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 11:58:04.608525 22916 tablet.cc:2369] T 1ccbdfc56ca64bb2b2bb91d14877347b P fa812b7ff43f40a7b6e2dc0c5fad925a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:58:04.630373 22935 raft_consensus.cc:482] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:58:04.630671 22935 raft_consensus.cc:504] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed0ea19b39a461887cb746015b9d779" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43291 } } peers { permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 } } peers { permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 } }
I20260811 11:58:04.631922 22935 leader_election.cc:288] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa812b7ff43f40a7b6e2dc0c5fad925a (127.6.143.131:41443), e449f197ad66458097ce41adc60d11b0 (127.6.143.129:42901)
I20260811 11:58:04.637782 22926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:04.632468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:46353 (local address 127.6.143.131:41443)
0811 11:58:04.632587 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:04.632591 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:04.632608 (+ 17us) server_negotiation.cc:407] Connection header received
0811 11:58:04.632669 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:04.632674 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:04.632761 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:04.632868 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:58:04.633535 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:04.634358 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:04.636368 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:04.636972 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:04.637122 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:04.637175 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:04.637567 (+ 392us) server_negotiation.cc:299] Negotiation successful
0811 11:58:04.637642 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 11:58:04.637786 22936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:04.632311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:60229 (local address 127.6.143.129:42901)
0811 11:58:04.632663 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:04.632667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:04.632684 (+ 17us) server_negotiation.cc:407] Connection header received
0811 11:58:04.633021 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:04.633024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:04.633081 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:04.633226 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:58:04.633473 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:04.634358 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:04.636368 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:04.637033 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:04.637146 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:04.637312 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:04.637567 (+ 255us) server_negotiation.cc:299] Negotiation successful
0811 11:58:04.637648 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":120,"threads_started":1}
I20260811 11:58:04.637820 22773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:04.632207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:41443 (local address 127.6.143.130:46353)
0811 11:58:04.632314 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:04.632330 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:04.632432 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:04.632894 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:04.632897 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:04.632913 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:58:04.633061 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:04.633068 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:04.634485 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:04.634489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:04.636216 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:04.636228 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:04.637111 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:04.637116 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:04.637361 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:04.637442 (+ 81us) client_negotiation.cc:769] Sending connection context
0811 11:58:04.637526 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 11:58:04.637656 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":328}
I20260811 11:58:04.637828 22937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:04.632207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:42901 (local address 127.6.143.130:60229)
0811 11:58:04.632619 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:04.632633 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:04.632718 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:04.633237 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:04.633240 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:04.633255 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:58:04.633341 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:04.633345 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:04.634485 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:04.634489 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:04.636216 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:04.636228 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:04.637135 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:04.637137 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:04.637401 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:04.637444 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 11:58:04.637526 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 11:58:04.637656 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":452,"thread_start_us":82,"threads_started":1}
I20260811 11:58:04.638492 22862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ccbdfc56ca64bb2b2bb91d14877347b" candidate_uuid: "2ed0ea19b39a461887cb746015b9d779" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" is_pre_election: true
I20260811 11:58:04.638489 22568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ccbdfc56ca64bb2b2bb91d14877347b" candidate_uuid: "2ed0ea19b39a461887cb746015b9d779" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e449f197ad66458097ce41adc60d11b0" is_pre_election: true
I20260811 11:58:04.638813 22862 raft_consensus.cc:2454] T 1ccbdfc56ca64bb2b2bb91d14877347b P fa812b7ff43f40a7b6e2dc0c5fad925a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ed0ea19b39a461887cb746015b9d779 in term 0.
I20260811 11:58:04.638829 22568 raft_consensus.cc:2454] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2ed0ea19b39a461887cb746015b9d779 in term 0.
I20260811 11:58:04.639349 22649 leader_election.cc:302] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2ed0ea19b39a461887cb746015b9d779, e449f197ad66458097ce41adc60d11b0, fa812b7ff43f40a7b6e2dc0c5fad925a; no voters:
I20260811 11:58:04.639619 22935 raft_consensus.cc:2781] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:58:04.639698 22935 raft_consensus.cc:482] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:58:04.639730 22935 raft_consensus.cc:3037] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:58:04.640978 22935 raft_consensus.cc:504] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed0ea19b39a461887cb746015b9d779" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43291 } } peers { permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 } } peers { permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 } }
I20260811 11:58:04.641335 22935 leader_election.cc:288] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779 [CANDIDATE]: Term 1 election: Requested vote from peers fa812b7ff43f40a7b6e2dc0c5fad925a (127.6.143.131:41443), e449f197ad66458097ce41adc60d11b0 (127.6.143.129:42901)
I20260811 11:58:04.641744 22862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ccbdfc56ca64bb2b2bb91d14877347b" candidate_uuid: "2ed0ea19b39a461887cb746015b9d779" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a"
I20260811 11:58:04.641748 22568 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1ccbdfc56ca64bb2b2bb91d14877347b" candidate_uuid: "2ed0ea19b39a461887cb746015b9d779" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e449f197ad66458097ce41adc60d11b0"
I20260811 11:58:04.641873 22568 raft_consensus.cc:3037] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:58:04.641888 22862 raft_consensus.cc:3037] T 1ccbdfc56ca64bb2b2bb91d14877347b P fa812b7ff43f40a7b6e2dc0c5fad925a [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:58:04.643141 22862 raft_consensus.cc:2454] T 1ccbdfc56ca64bb2b2bb91d14877347b P fa812b7ff43f40a7b6e2dc0c5fad925a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ed0ea19b39a461887cb746015b9d779 in term 1.
I20260811 11:58:04.643141 22568 raft_consensus.cc:2454] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2ed0ea19b39a461887cb746015b9d779 in term 1.
I20260811 11:58:04.643450 22646 leader_election.cc:302] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ed0ea19b39a461887cb746015b9d779, fa812b7ff43f40a7b6e2dc0c5fad925a; no voters:
I20260811 11:58:04.643635 22935 raft_consensus.cc:2781] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:58:04.643949 22935 raft_consensus.cc:686] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779 [term 1 LEADER]: Becoming Leader. State: Replica: 2ed0ea19b39a461887cb746015b9d779, State: Running, Role: LEADER
I20260811 11:58:04.644510 22935 consensus_queue.cc:237] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed0ea19b39a461887cb746015b9d779" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43291 } } peers { permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 } } peers { permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 } }
I20260811 11:58:04.647578 22405 catalog_manager.cc:5495] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2ed0ea19b39a461887cb746015b9d779 (127.6.143.130). New cstate: current_term: 1 leader_uuid: "2ed0ea19b39a461887cb746015b9d779" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed0ea19b39a461887cb746015b9d779" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 } health_report { overall_health: UNKNOWN } } }
W20260811 11:58:04.664846 22772 tablet.cc:2369] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:58:04.698915 22924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:04.695215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41446 (local address 127.6.143.130:43291)
0811 11:58:04.695414 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:04.695418 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:04.695433 (+ 15us) server_negotiation.cc:407] Connection header received
0811 11:58:04.695585 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:04.695588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:04.695648 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:04.695743 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:58:04.696004 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:04.696697 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:04.697549 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:04.697756 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:04.697833 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:04.698121 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:58:04.698521 (+ 400us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:58:04.698610 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:04.698667 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 11:58:04.698713 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 11:58:04.704828 22862 raft_consensus.cc:1261] T 1ccbdfc56ca64bb2b2bb91d14877347b P fa812b7ff43f40a7b6e2dc0c5fad925a [term 1 FOLLOWER]: Refusing update from remote peer 2ed0ea19b39a461887cb746015b9d779: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:58:04.704824 22568 raft_consensus.cc:1261] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [term 1 FOLLOWER]: Refusing update from remote peer 2ed0ea19b39a461887cb746015b9d779: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:58:04.705586 22935 consensus_queue.cc:1035] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:58:04.705799 22938 consensus_queue.cc:1035] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:58:04.712179 22941 mvcc.cc:205] Tried to move back new op lower bound from 7317297089342906368 to 7317297089106546688. Current Snapshot: MvccSnapshot[applied={T|T < 7317297089342906368}]
I20260811 11:58:04.712251 22940 mvcc.cc:205] Tried to move back new op lower bound from 7317297089342906368 to 7317297089106546688. Current Snapshot: MvccSnapshot[applied={T|T < 7317297089342906368}]
I20260811 11:58:04.713920 22942 mvcc.cc:205] Tried to move back new op lower bound from 7317297089342906368 to 7317297089106546688. Current Snapshot: MvccSnapshot[applied={T|T < 7317297089342906368}]
I20260811 11:58:04.722266 22405 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:36426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:58:04.722399 22405 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:36426:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:58:04.724686 22405 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710: Sending DeleteTablet for 3 replicas of tablet 1ccbdfc56ca64bb2b2bb91d14877347b
I20260811 11:58:04.725425 22841 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ccbdfc56ca64bb2b2bb91d14877347b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:58:04 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:55162
I20260811 11:58:04.725447 22697 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ccbdfc56ca64bb2b2bb91d14877347b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:58:04 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:41432
I20260811 11:58:04.725617 22547 tablet_service.cc:1514] Processing DeleteTablet for tablet 1ccbdfc56ca64bb2b2bb91d14877347b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:58:04 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:58668
I20260811 11:58:04.725934 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 22451
I20260811 11:58:04.725963 22950 tablet_replica.cc:330] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779: stopping tablet replica
I20260811 11:58:04.726257 22950 raft_consensus.cc:2229] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779 [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:58:04.726878 22950 raft_consensus.cc:2258] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:58:04.729398 22949 tablet_replica.cc:330] T 1ccbdfc56ca64bb2b2bb91d14877347b P fa812b7ff43f40a7b6e2dc0c5fad925a: stopping tablet replica
I20260811 11:58:04.729719 22949 raft_consensus.cc:2229] T 1ccbdfc56ca64bb2b2bb91d14877347b P fa812b7ff43f40a7b6e2dc0c5fad925a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:58:04.730043 22949 raft_consensus.cc:2258] T 1ccbdfc56ca64bb2b2bb91d14877347b P fa812b7ff43f40a7b6e2dc0c5fad925a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:58:04.731508 22950 ts_tablet_manager.cc:1905] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:58:04.733829 22949 ts_tablet_manager.cc:1905] T 1ccbdfc56ca64bb2b2bb91d14877347b P fa812b7ff43f40a7b6e2dc0c5fad925a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:58:04.734264 22950 ts_tablet_manager.cc:1918] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:58:04.734423 22950 log.cc:1196] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/wal/wals/1ccbdfc56ca64bb2b2bb91d14877347b
I20260811 11:58:04.734824 22950 ts_tablet_manager.cc:1939] T 1ccbdfc56ca64bb2b2bb91d14877347b P 2ed0ea19b39a461887cb746015b9d779: Deleting consensus metadata
I20260811 11:58:04.736508 22389 catalog_manager.cc:4841] TS 2ed0ea19b39a461887cb746015b9d779 (127.6.143.130:43291): tablet 1ccbdfc56ca64bb2b2bb91d14877347b (table test-table [id=9a5f1bab55144315b3baa3fc3da50e51]) successfully deleted
I20260811 11:58:04.737329 22949 ts_tablet_manager.cc:1918] T 1ccbdfc56ca64bb2b2bb91d14877347b P fa812b7ff43f40a7b6e2dc0c5fad925a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:58:04.737447 22949 log.cc:1196] T 1ccbdfc56ca64bb2b2bb91d14877347b P fa812b7ff43f40a7b6e2dc0c5fad925a: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/wal/wals/1ccbdfc56ca64bb2b2bb91d14877347b
W20260811 11:58:04.737672 22392 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.143.129:42901 (error 108)
I20260811 11:58:04.737810 22949 ts_tablet_manager.cc:1939] T 1ccbdfc56ca64bb2b2bb91d14877347b P fa812b7ff43f40a7b6e2dc0c5fad925a: Deleting consensus metadata
W20260811 11:58:04.738940 22925 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:58:04.738632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:42901 (local address 127.0.0.1:58672)
0811 11:58:04.738778 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:04.738846 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.129:42901: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 11:58:04.739091 22389 catalog_manager.cc:4841] TS fa812b7ff43f40a7b6e2dc0c5fad925a (127.6.143.131:41443): tablet 1ccbdfc56ca64bb2b2bb91d14877347b (table test-table [id=9a5f1bab55144315b3baa3fc3da50e51]) successfully deleted
I20260811 11:58:04.739151 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 22624
W20260811 11:58:04.739147 22392 catalog_manager.cc:4584] TS e449f197ad66458097ce41adc60d11b0 (127.6.143.129:42901): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1ccbdfc56ca64bb2b2bb91d14877347b: Network error: Client connection negotiation failed: client connection to 127.6.143.129:42901: connect: Connection reset by peer (error 104)
I20260811 11:58:04.747529 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 22774
I20260811 11:58:04.754892 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 22371
I20260811 11:58:04.762936 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:39315
--webserver_interface=127.6.143.190
--webserver_port=45187
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:43795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:39315
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:58:04.873809 22953 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:58:04.874037 22953 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:58:04.874123 22953 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:58:04.878134 22953 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:58:04.878183 22953 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:58:04.878225 22953 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:58:04.878262 22953 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:58:04.878294 22953 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:58:04.883435 22953 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:43795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:39315
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:39315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=45187
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22953
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:58:04.884469 22953 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:58:04.885423 22953 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:58:04.893697 22961 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:58:04.893785 22953 server_base.cc:1034] running on GCE node
W20260811 11:58:04.893714 22959 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:04.893700 22958 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:58:04.894604 22953 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:58:04.895625 22953 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:58:04.896813 22953 hybrid_clock.cc:648] HybridClock initialized: now 1786449484896780 us; error 45 us; skew 500 ppm
Aug 11 11:58:04 dist-test-slave-zkpd krb5kdc[21765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449484, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:58:04.899561 22953 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:58:04.900451 22953 webserver.cc:460] Webserver started at http://127.6.143.190:45187/ using document root <none> and password file <none>
I20260811 11:58:04.900943 22953 fs_manager.cc:359] Metadata directory not provided
I20260811 11:58:04.901001 22953 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:58:04.904548 22953 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:58:04.906505 22968 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:04.907646 22953 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:58:04.907742 22953 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "6976882a3ff84562a7f6cdf18f466710"
format_stamp: "Formatted at 2026-08-11 11:58:02 on dist-test-slave-zkpd"
server_key: "3575d3614599cad6c8a56e681d140a07"
server_key_iv: "dcedca16a62105884842c8fe10d7e5c5"
server_key_version: "encryptionkey@0"
I20260811 11:58:04.908305 22953 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:58:04.922196 22953 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:58:04.925237 22953 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:58:04.925369 22953 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:58:04.932250 22953 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:39315
I20260811 11:58:04.932562 23024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:39315 every 8 connection(s)
I20260811 11:58:04.933285 22953 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:58:04.937481 23025 sys_catalog.cc:263] Verifying existing consensus state
I20260811 11:58:04.939100 23025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710: Bootstrap starting.
I20260811 11:58:04.939468 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 22953
I20260811 11:58:04.939924 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:42901
--local_ip_for_outbound_sockets=127.6.143.129
--tserver_master_addrs=127.6.143.190:39315
--webserver_port=38909
--webserver_interface=127.6.143.129
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:43795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:58:04.957002 23025 log.cc:824] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710: Log is configured to *not* fsync() on all Append() calls
I20260811 11:58:04.968240 23025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 11:58:04.968628 23025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710: Bootstrap complete.
I20260811 11:58:04.973061 23025 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6976882a3ff84562a7f6cdf18f466710" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39315 } }
I20260811 11:58:04.973457 23025 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6976882a3ff84562a7f6cdf18f466710, State: Initialized, Role: FOLLOWER
I20260811 11:58:04.973982 23025 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6976882a3ff84562a7f6cdf18f466710" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39315 } }
I20260811 11:58:04.974097 23025 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:58:04.974188 23025 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:58:04.974305 23025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [term 2 FOLLOWER]: Advancing to term 3
I20260811 11:58:04.976310 23025 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6976882a3ff84562a7f6cdf18f466710" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39315 } }
I20260811 11:58:04.976642 23025 leader_election.cc:302] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6976882a3ff84562a7f6cdf18f466710; no voters:
I20260811 11:58:04.977047 23025 leader_election.cc:288] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 11:58:04.977131 23029 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 11:58:04.977474 23029 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [term 3 LEADER]: Becoming Leader. State: Replica: 6976882a3ff84562a7f6cdf18f466710, State: Running, Role: LEADER
I20260811 11:58:04.977815 23029 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6976882a3ff84562a7f6cdf18f466710" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39315 } }
I20260811 11:58:04.978343 23025 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:58:04.978812 23029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6976882a3ff84562a7f6cdf18f466710. Latest consensus state: current_term: 3 leader_uuid: "6976882a3ff84562a7f6cdf18f466710" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6976882a3ff84562a7f6cdf18f466710" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39315 } } }
I20260811 11:58:04.978926 23029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [sys.catalog]: This master's current role is: LEADER
I20260811 11:58:04.979329 23030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "6976882a3ff84562a7f6cdf18f466710" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6976882a3ff84562a7f6cdf18f466710" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39315 } } }
I20260811 11:58:04.979496 23030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6976882a3ff84562a7f6cdf18f466710 [sys.catalog]: This master's current role is: LEADER
I20260811 11:58:04.979717 23034 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:58:04.984896 23034 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:58:04.985623 23034 catalog_manager.cc:1259] Loaded cluster ID: 55aabba2cb2c43a28ccb489252d4eb3c
I20260811 11:58:04.985700 23034 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 11:58:04.986559 23034 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fdea0b3b9ad google::LogMessage::Fail() at ??:0
@ 0x7fdea0b3e88a google::LogMessage::SendToLog() at ??:0
@ 0x7fdea0b3b4de google::LogMessage::Flush() at ??:0
@ 0x7fdea0b3ca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fdeab47afdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fdeab47bd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fdeab478579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fdeab4bc640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fdea2d8d3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fdea2d8dc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fdea2d8f3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fdea2d7e512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fdea43c26db start_thread at ??:0
@ 0x7fde9f63961f clone at ??:0
W20260811 11:58:05.068835 23027 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:58:05.069095 23027 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:58:05.069200 23027 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:58:05.073216 23027 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:58:05.073288 23027 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:58:05.073395 23027 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:58:05.078687 23027 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:43795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:42901
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=38909
--tserver_master_addrs=127.6.143.190:39315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23027
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:58:05.079685 23027 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:58:05.080725 23027 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:58:05.090143 23068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:05.090138 23067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:58:05.090384 23027 server_base.cc:1034] running on GCE node
W20260811 11:58:05.090615 23070 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:58:05.091050 23027 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:58:05.091661 23027 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:58:05.092839 23027 hybrid_clock.cc:648] HybridClock initialized: now 1786449485092815 us; error 38 us; skew 500 ppm
Aug 11 11:58:05 dist-test-slave-zkpd krb5kdc[21765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449485, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:58:05.095517 23027 init.cc:377] Logged in from keytab as kudu/127.6.143.129@KRBTEST.COM (short username kudu)
I20260811 11:58:05.096432 23027 webserver.cc:460] Webserver started at http://127.6.143.129:38909/ using document root <none> and password file <none>
I20260811 11:58:05.096987 23027 fs_manager.cc:359] Metadata directory not provided
I20260811 11:58:05.097048 23027 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:58:05.101075 23027 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 11:58:05.103408 23079 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:05.104768 23027 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 11:58:05.104897 23027 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "e449f197ad66458097ce41adc60d11b0"
format_stamp: "Formatted at 2026-08-11 11:58:02 on dist-test-slave-zkpd"
server_key: "e5cf7993e726d5e81052a82582aac991"
server_key_iv: "70affad25a3f9fa2e6d63c45f5c84c6e"
server_key_version: "encryptionkey@0"
I20260811 11:58:05.105425 23027 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:58:05.125000 23027 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:58:05.128271 23027 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:58:05.128463 23027 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:58:05.129048 23027 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:58:05.130154 23086 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 11:58:05.135522 23027 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 11:58:05.135601 23027 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260811 11:58:05.135699 23027 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 11:58:05.137444 23027 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 11:58:05.137511 23027 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 11:58:05.137807 23086 tablet_bootstrap.cc:492] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0: Bootstrap starting.
I20260811 11:58:05.148524 23027 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:42901
I20260811 11:58:05.148594 23205 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:42901 every 8 connection(s)
I20260811 11:58:05.149493 23027 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:58:05.157739 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 23027
I20260811 11:58:05.158215 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:43291
--local_ip_for_outbound_sockets=127.6.143.130
--tserver_master_addrs=127.6.143.190:39315
--webserver_port=41815
--webserver_interface=127.6.143.130
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:43795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:58:05.159773 23086 log.cc:824] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0: Log is configured to *not* fsync() on all Append() calls
I20260811 11:58:05.167573 23086 tablet_bootstrap.cc:492] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 11:58:05.168036 23086 tablet_bootstrap.cc:492] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0: Bootstrap complete.
I20260811 11:58:05.169229 23086 ts_tablet_manager.cc:1397] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0: Time spent bootstrapping tablet: real 0.032s user 0.018s sys 0.012s
I20260811 11:58:05.171696 23086 raft_consensus.cc:348] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed0ea19b39a461887cb746015b9d779" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43291 } } peers { permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 } } peers { permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 } }
I20260811 11:58:05.172204 23086 raft_consensus.cc:729] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e449f197ad66458097ce41adc60d11b0, State: Initialized, Role: FOLLOWER
I20260811 11:58:05.172755 23086 consensus_queue.cc:260] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed0ea19b39a461887cb746015b9d779" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43291 } } peers { permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 } } peers { permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 } }
I20260811 11:58:05.173751 23086 ts_tablet_manager.cc:1428] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
W20260811 11:58:05.228075 23208 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:58:05.151436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.129:60209)
0811 11:58:05.151960 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:05.152014 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:05.152854 (+ 840us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:05.227250 (+ 74396us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":144,"threads_started":1}
W20260811 11:58:05.228703 23206 heartbeater.cc:644] Failed to heartbeat to 127.6.143.190:39315 (0 consecutive failures): Network error: Failed to ping master at 127.6.143.190:39315: Client connection negotiation failed: client connection to 127.6.143.190:39315: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 11:58:05.229449 23208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:05.229183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.129:52507)
0811 11:58:05.229325 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:05.229396 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 11:58:05.230177 23208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:05.229978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.129:56333)
0811 11:58:05.230081 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:05.230134 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 11:58:05.230376 23206 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 11:58:05.296298 23209 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:58:05.296597 23209 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:58:05.296684 23209 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:58:05.300725 23209 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:58:05.300819 23209 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:58:05.300908 23209 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:58:05.306167 23209 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:43795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:43291
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=41815
--tserver_master_addrs=127.6.143.190:39315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23209
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:58:05.307217 23209 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:58:05.308285 23209 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:58:05.318061 23219 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:05.318161 23217 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:05.318065 23216 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:58:05.318274 23209 server_base.cc:1034] running on GCE node
I20260811 11:58:05.318819 23209 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:58:05.319553 23209 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:58:05.320753 23209 hybrid_clock.cc:648] HybridClock initialized: now 1786449485320696 us; error 70 us; skew 500 ppm
Aug 11 11:58:05 dist-test-slave-zkpd krb5kdc[21765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449485, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:58:05.323740 23209 init.cc:377] Logged in from keytab as kudu/127.6.143.130@KRBTEST.COM (short username kudu)
I20260811 11:58:05.324877 23209 webserver.cc:460] Webserver started at http://127.6.143.130:41815/ using document root <none> and password file <none>
I20260811 11:58:05.325618 23209 fs_manager.cc:359] Metadata directory not provided
I20260811 11:58:05.325719 23209 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:58:05.330124 23209 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 11:58:05.332293 23228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:05.333477 23209 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:58:05.333613 23209 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "2ed0ea19b39a461887cb746015b9d779"
format_stamp: "Formatted at 2026-08-11 11:58:02 on dist-test-slave-zkpd"
server_key: "3708bd5c0a8e756af7ff84b01ce46a26"
server_key_iv: "0fad8a6cffa5babb916453ffa4b9eaba"
server_key_version: "encryptionkey@0"
I20260811 11:58:05.334179 23209 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:58:05.352088 23209 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:58:05.355520 23209 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:58:05.355733 23209 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:58:05.356371 23209 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:58:05.357208 23209 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:58:05.357255 23209 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:05.357327 23209 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:58:05.357368 23209 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:05.366184 23209 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:43291
I20260811 11:58:05.366256 23354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:43291 every 8 connection(s)
I20260811 11:58:05.367225 23209 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
I20260811 11:58:05.369266 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 23209
I20260811 11:58:05.369678 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:41443
--local_ip_for_outbound_sockets=127.6.143.131
--tserver_master_addrs=127.6.143.190:39315
--webserver_port=37761
--webserver_interface=127.6.143.131
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:43795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:58:05.371366 23357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:05.369508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.130:53425)
0811 11:58:05.370000 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:05.370562 (+ 562us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":137,"threads_started":1}
W20260811 11:58:05.371883 23355 heartbeater.cc:644] Failed to heartbeat to 127.6.143.190:39315 (0 consecutive failures): Network error: Failed to ping master at 127.6.143.190:39315: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
I20260811 11:58:05.372619 23357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:05.372320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.130:50365)
0811 11:58:05.372458 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:05.372526 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 11:58:05.373370 23357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:05.373087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.130:47873)
0811 11:58:05.373222 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:05.373282 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 11:58:05.373557 23355 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 11:58:05.488952 23358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:58:05.489182 23358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:58:05.489246 23358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:58:05.493165 23358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:58:05.493221 23358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:58:05.493276 23358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:58:05.498792 23358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:43795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:41443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=37761
--tserver_master_addrs=127.6.143.190:39315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:58:05.499785 23358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:58:05.500846 23358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:58:05.510695 23366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:05.510733 23363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:05.510743 23364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:58:05.510877 23358 server_base.cc:1034] running on GCE node
I20260811 11:58:05.511236 23358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:58:05.511858 23358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:58:05.513074 23358 hybrid_clock.cc:648] HybridClock initialized: now 1786449485513054 us; error 30 us; skew 500 ppm
Aug 11 11:58:05 dist-test-slave-zkpd krb5kdc[21765](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449485, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:58:05.515969 23358 init.cc:377] Logged in from keytab as kudu/127.6.143.131@KRBTEST.COM (short username kudu)
I20260811 11:58:05.516850 23358 webserver.cc:460] Webserver started at http://127.6.143.131:37761/ using document root <none> and password file <none>
I20260811 11:58:05.517412 23358 fs_manager.cc:359] Metadata directory not provided
I20260811 11:58:05.517506 23358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:58:05.521322 23358 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 11:58:05.523313 23374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:05.524592 23358 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:58:05.524725 23358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a"
format_stamp: "Formatted at 2026-08-11 11:58:03 on dist-test-slave-zkpd"
server_key: "216578dd81d975c716227214f1b037d2"
server_key_iv: "2f8b6b94ed337e4fd859c3c296519c55"
server_key_version: "encryptionkey@0"
I20260811 11:58:05.525281 23358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:58:05.543682 23358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:58:05.546504 23358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:58:05.546707 23358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:58:05.547277 23358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:58:05.548168 23358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:58:05.548236 23358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:05.548307 23358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:58:05.548358 23358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:05.557327 23358 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:41443
I20260811 11:58:05.557366 23501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:41443 every 8 connection(s)
I20260811 11:58:05.558310 23358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:58:05.562125 23505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:05.560426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.131:53795)
0811 11:58:05.561325 (+ 899us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:05.561439 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":703,"thread_start_us":552,"threads_started":1}
W20260811 11:58:05.562640 23503 heartbeater.cc:644] Failed to heartbeat to 127.6.143.190:39315 (0 consecutive failures): Network error: Failed to ping master at 127.6.143.190:39315: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
I20260811 11:58:05.563215 23505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:05.563011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.131:43159)
0811 11:58:05.563115 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:05.563161 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 11:58:05.563942 23505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:05.563682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.131:58569)
0811 11:58:05.563786 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:05.563833 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 11:58:05.564142 23503 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 11:58:05.567790 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 23358
I20260811 11:58:06.231312 23507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:06.230897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.129:56087)
0811 11:58:06.231148 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:06.231201 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":74,"threads_started":1}
I20260811 11:58:06.374444 23508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:06.374070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.130:51863)
0811 11:58:06.374297 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:06.374347 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":66,"threads_started":1}
I20260811 11:58:06.521886 23509 raft_consensus.cc:482] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:58:06.522079 23509 raft_consensus.cc:504] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed0ea19b39a461887cb746015b9d779" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43291 } } peers { permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 } } peers { permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 } }
I20260811 11:58:06.523105 23509 leader_election.cc:288] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ed0ea19b39a461887cb746015b9d779 (127.6.143.130:43291), fa812b7ff43f40a7b6e2dc0c5fad925a (127.6.143.131:41443)
Aug 11 11:58:06 dist-test-slave-zkpd krb5kdc[21765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449485, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for kudu/127.6.143.131@KRBTEST.COM
Aug 11 11:58:06 dist-test-slave-zkpd krb5kdc[21765](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449485, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for kudu/127.6.143.130@KRBTEST.COM
I20260811 11:58:06.542568 23507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:06.523338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:43291 (local address 127.6.143.129:55527)
0811 11:58:06.523473 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:06.523495 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:06.523617 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:06.525802 (+ 2185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:06.525808 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:06.526371 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:58:06.526815 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:06.526828 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:06.528303 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:06.528306 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:06.529834 (+ 1528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:06.529846 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:06.530526 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:06.530529 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:06.530660 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:06.535950 (+ 5290us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:58:06.535970 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:58:06.538088 (+ 2118us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:58:06.540495 (+ 2407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:06.540501 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:06.540504 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:06.540759 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:06.541152 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:06.541154 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:06.541155 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:06.541226 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:06.541897 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:58:06.541911 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:58:06.542000 (+ 89us) client_negotiation.cc:769] Sending connection context
0811 11:58:06.542096 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 11:58:06.542403 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43,"mutex_wait_us":341,"spinlock_wait_cycles":3840}
I20260811 11:58:06.542671 23512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:06.523801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:52353 (local address 127.6.143.131:41443)
0811 11:58:06.524309 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:06.524326 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:06.524377 (+ 51us) server_negotiation.cc:407] Connection header received
0811 11:58:06.525097 (+ 720us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:06.525104 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:06.525441 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:06.525750 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:58:06.526988 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:06.528150 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:06.530668 (+ 2518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:06.531102 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:06.531270 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:06.537918 (+ 6648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:58:06.537923 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:58:06.537936 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:58:06.537964 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:58:06.539802 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:06.540439 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:06.540444 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:06.540448 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:06.540558 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:06.540955 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:06.540958 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:06.540960 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:06.541481 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:58:06.541617 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:06.542185 (+ 568us) server_negotiation.cc:299] Negotiation successful
0811 11:58:06.542478 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":153,"threads_started":1}
I20260811 11:58:06.542672 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:06.523545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:55527 (local address 127.6.143.130:43291)
0811 11:58:06.524001 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:06.524023 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:06.524100 (+ 77us) server_negotiation.cc:407] Connection header received
0811 11:58:06.524994 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:06.525002 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:06.525442 (+ 440us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:06.525750 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:58:06.527010 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:06.528063 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:06.529954 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:06.530418 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:06.530541 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:06.538299 (+ 7758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:58:06.538309 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:58:06.538324 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:58:06.538359 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:58:06.540413 (+ 2054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:06.540872 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:06.540877 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:06.540880 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:06.541048 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:06.541312 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:06.541316 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:06.541317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:06.541779 (+ 462us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:58:06.541869 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:06.542218 (+ 349us) server_negotiation.cc:299] Negotiation successful
0811 11:58:06.542478 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":131,"threads_started":1}
I20260811 11:58:06.542567 23511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:06.523620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:41443 (local address 127.6.143.129:52353)
0811 11:58:06.523919 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:06.523935 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:06.524034 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:06.525802 (+ 1768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:06.525808 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:06.526375 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:58:06.526815 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:06.526828 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:06.528312 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:06.528315 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:06.529834 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:06.529846 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:06.531229 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:06.531233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:06.531318 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:06.535967 (+ 4649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:58:06.535980 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:58:06.537674 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:58:06.539956 (+ 2282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:06.539962 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:06.539977 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:06.540324 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:06.540657 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:06.540663 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:06.540665 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:06.540843 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:06.541596 (+ 753us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:58:06.541601 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:58:06.541802 (+ 201us) client_negotiation.cc:769] Sending connection context
0811 11:58:06.542009 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 11:58:06.542353 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"mutex_wait_us":124,"spinlock_wait_cycles":25728,"thread_start_us":65,"threads_started":1}
W20260811 11:58:06.545238 23080 leader_election.cc:341] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fa812b7ff43f40a7b6e2dc0c5fad925a (127.6.143.131:41443): Not found: Tablet not found: 1ccbdfc56ca64bb2b2bb91d14877347b
W20260811 11:58:06.545451 23080 leader_election.cc:341] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ed0ea19b39a461887cb746015b9d779 (127.6.143.130:43291): Not found: Tablet not found: 1ccbdfc56ca64bb2b2bb91d14877347b
I20260811 11:58:06.545527 23080 leader_election.cc:302] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e449f197ad66458097ce41adc60d11b0; no voters: 2ed0ea19b39a461887cb746015b9d779, fa812b7ff43f40a7b6e2dc0c5fad925a
I20260811 11:58:06.545815 23509 raft_consensus.cc:2726] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 11:58:06.564993 23513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:06.564636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.131:44703)
0811 11:58:06.564852 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:06.564909 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":64,"threads_started":1}
I20260811 11:58:07.232577 23514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:07.232160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.129:50191)
0811 11:58:07.232426 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:07.232480 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":74,"threads_started":1}
I20260811 11:58:07.375439 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:07.375080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.130:37027)
0811 11:58:07.375312 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:07.375359 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":72,"threads_started":1}
I20260811 11:58:07.566179 23516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:07.565788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.131:42183)
0811 11:58:07.566014 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:07.566084 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":68,"threads_started":1}
I20260811 11:58:08.144747 23518 raft_consensus.cc:482] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:58:08.145013 23518 raft_consensus.cc:504] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed0ea19b39a461887cb746015b9d779" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43291 } } peers { permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 } } peers { permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 } }
I20260811 11:58:08.145454 23518 leader_election.cc:288] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ed0ea19b39a461887cb746015b9d779 (127.6.143.130:43291), fa812b7ff43f40a7b6e2dc0c5fad925a (127.6.143.131:41443)
W20260811 11:58:08.146327 23080 leader_election.cc:341] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ed0ea19b39a461887cb746015b9d779 (127.6.143.130:43291): Not found: Tablet not found: 1ccbdfc56ca64bb2b2bb91d14877347b
W20260811 11:58:08.146484 23080 leader_election.cc:341] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fa812b7ff43f40a7b6e2dc0c5fad925a (127.6.143.131:41443): Not found: Tablet not found: 1ccbdfc56ca64bb2b2bb91d14877347b
I20260811 11:58:08.146545 23080 leader_election.cc:302] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e449f197ad66458097ce41adc60d11b0; no voters: 2ed0ea19b39a461887cb746015b9d779, fa812b7ff43f40a7b6e2dc0c5fad925a
I20260811 11:58:08.146727 23518 raft_consensus.cc:2726] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 11:58:08.233770 23520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:08.233342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.129:60041)
0811 11:58:08.233611 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:08.233664 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":91,"threads_started":1}
I20260811 11:58:08.376780 23521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:08.376307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.130:60751)
0811 11:58:08.376566 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:08.376681 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":93,"threads_started":1}
I20260811 11:58:08.567322 23523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:08.566944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.131:42113)
0811 11:58:08.567186 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:08.567238 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":76,"threads_started":1}
I20260811 11:58:09.234946 23524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:09.234552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.129:39937)
0811 11:58:09.234790 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:09.234842 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":76,"threads_started":1}
I20260811 11:58:09.377969 23525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:09.377555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.130:43389)
0811 11:58:09.377821 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:09.377882 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":91,"threads_started":1}
I20260811 11:58:09.568418 23526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:09.568027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.131:57431)
0811 11:58:09.568260 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:09.568314 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":69,"threads_started":1}
I20260811 11:58:10.236330 23527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:10.235817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.129:33695)
0811 11:58:10.236171 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:10.236231 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":125,"threads_started":1}
I20260811 11:58:10.348440 23528 raft_consensus.cc:482] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:58:10.348567 23528 raft_consensus.cc:504] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed0ea19b39a461887cb746015b9d779" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43291 } } peers { permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 } } peers { permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 } }
I20260811 11:58:10.348917 23528 leader_election.cc:288] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ed0ea19b39a461887cb746015b9d779 (127.6.143.130:43291), fa812b7ff43f40a7b6e2dc0c5fad925a (127.6.143.131:41443)
W20260811 11:58:10.349725 23080 leader_election.cc:341] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ed0ea19b39a461887cb746015b9d779 (127.6.143.130:43291): Not found: Tablet not found: 1ccbdfc56ca64bb2b2bb91d14877347b
W20260811 11:58:10.349877 23080 leader_election.cc:341] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fa812b7ff43f40a7b6e2dc0c5fad925a (127.6.143.131:41443): Not found: Tablet not found: 1ccbdfc56ca64bb2b2bb91d14877347b
I20260811 11:58:10.349947 23080 leader_election.cc:302] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e449f197ad66458097ce41adc60d11b0; no voters: 2ed0ea19b39a461887cb746015b9d779, fa812b7ff43f40a7b6e2dc0c5fad925a
I20260811 11:58:10.350121 23528 raft_consensus.cc:2726] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 11:58:10.379142 23529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:10.378726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.130:36391)
0811 11:58:10.378991 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:10.379049 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":76,"threads_started":1}
I20260811 11:58:10.569435 23531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:10.569075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.131:54551)
0811 11:58:10.569301 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:10.569351 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":69,"threads_started":1}
I20260811 11:58:11.237535 23532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:11.237120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.129:44701)
0811 11:58:11.237386 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:11.237440 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":94,"threads_started":1}
I20260811 11:58:11.380343 23533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:11.379872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.130:52205)
0811 11:58:11.380186 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:11.380248 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":73,"threads_started":1}
I20260811 11:58:11.570639 23534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:11.570221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.131:57287)
0811 11:58:11.570494 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:11.570549 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":91,"threads_started":1}
I20260811 11:58:12.238822 23535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:12.238364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.129:51875)
0811 11:58:12.238663 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:12.238723 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":79,"threads_started":1}
I20260811 11:58:12.381680 23536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:12.381226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.130:47753)
0811 11:58:12.381521 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:12.381588 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":77,"threads_started":1}
I20260811 11:58:12.571874 23539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:12.571444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.131:36707)
0811 11:58:12.571732 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:12.571785 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":71,"threads_started":1}
I20260811 11:58:13.240068 23540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:13.239567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.129:54175)
0811 11:58:13.239818 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:13.239969 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":79,"threads_started":1}
I20260811 11:58:13.382792 23541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:13.382322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.130:57065)
0811 11:58:13.382576 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:13.382661 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":88,"threads_started":1}
I20260811 11:58:13.573009 23542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:13.572643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.131:33867)
0811 11:58:13.572870 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:13.572923 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":69,"threads_started":1}
I20260811 11:58:14.241307 23543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:14.240812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.129:36431)
0811 11:58:14.241151 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:14.241210 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":123,"threads_started":1}
I20260811 11:58:14.384126 23545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:14.383660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.130:33499)
0811 11:58:14.383970 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:14.384029 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":84,"threads_started":1}
I20260811 11:58:14.574142 23546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:14.573746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.131:59379)
0811 11:58:14.573993 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:14.574047 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":78,"threads_started":1}
I20260811 11:58:15.090730 23547 raft_consensus.cc:482] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:58:15.090881 23547 raft_consensus.cc:504] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed0ea19b39a461887cb746015b9d779" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43291 } } peers { permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 } } peers { permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 } }
I20260811 11:58:15.091240 23547 leader_election.cc:288] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ed0ea19b39a461887cb746015b9d779 (127.6.143.130:43291), fa812b7ff43f40a7b6e2dc0c5fad925a (127.6.143.131:41443)
W20260811 11:58:15.092064 23080 leader_election.cc:341] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fa812b7ff43f40a7b6e2dc0c5fad925a (127.6.143.131:41443): Not found: Tablet not found: 1ccbdfc56ca64bb2b2bb91d14877347b
W20260811 11:58:15.092213 23080 leader_election.cc:341] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ed0ea19b39a461887cb746015b9d779 (127.6.143.130:43291): Not found: Tablet not found: 1ccbdfc56ca64bb2b2bb91d14877347b
I20260811 11:58:15.092308 23080 leader_election.cc:302] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e449f197ad66458097ce41adc60d11b0; no voters: 2ed0ea19b39a461887cb746015b9d779, fa812b7ff43f40a7b6e2dc0c5fad925a
I20260811 11:58:15.092496 23547 raft_consensus.cc:2726] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 11:58:15.242587 23548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:15.242224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.129:47779)
0811 11:58:15.242442 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:15.242492 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":66,"threads_started":1}
I20260811 11:58:15.385269 23549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:15.384881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.130:54157)
0811 11:58:15.385126 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:15.385180 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":67,"threads_started":1}
I20260811 11:58:15.575237 23550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:15.574834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.131:55417)
0811 11:58:15.575097 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:15.575151 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":86,"threads_started":1}
I20260811 11:58:16.243882 23551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:16.243327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.129:52461)
0811 11:58:16.243709 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:16.243767 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":86,"threads_started":1}
I20260811 11:58:16.386420 23552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:16.386017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.130:55971)
0811 11:58:16.386276 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:16.386328 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":92,"threads_started":1}
I20260811 11:58:16.576591 23553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:16.576112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.131:52709)
0811 11:58:16.576425 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:16.576482 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":123,"threads_started":1}
I20260811 11:58:17.245077 23554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:17.244663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.129:50289)
0811 11:58:17.244925 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:17.244981 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":81,"threads_started":1}
I20260811 11:58:17.387691 23555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:17.387219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.130:35355)
0811 11:58:17.387480 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:17.387560 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":99,"threads_started":1}
I20260811 11:58:17.577840 23556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:17.577359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.131:50721)
0811 11:58:17.577680 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:17.577744 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":85,"threads_started":1}
I20260811 11:58:18.246301 23557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:18.245870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.129:44193)
0811 11:58:18.246142 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:18.246202 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":76,"threads_started":1}
I20260811 11:58:18.388856 23558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:18.388466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.130:55333)
0811 11:58:18.388708 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:18.388762 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":75,"threads_started":1}
I20260811 11:58:18.579013 23559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:18.578598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.131:33239)
0811 11:58:18.578850 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:18.578916 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":74,"threads_started":1}
I20260811 11:58:19.247452 23560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:19.247049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.129:48515)
0811 11:58:19.247288 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:19.247340 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":74,"threads_started":1}
I20260811 11:58:19.389940 23561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:19.389549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.130:44265)
0811 11:58:19.389784 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:19.389837 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":76,"threads_started":1}
I20260811 11:58:19.580263 23562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:19.579815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.131:47339)
0811 11:58:19.580115 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:19.580169 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":136,"threads_started":1}
I20260811 11:58:20.001365 23563 raft_consensus.cc:482] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:58:20.001504 23563 raft_consensus.cc:504] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ed0ea19b39a461887cb746015b9d779" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43291 } } peers { permanent_uuid: "fa812b7ff43f40a7b6e2dc0c5fad925a" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 41443 } } peers { permanent_uuid: "e449f197ad66458097ce41adc60d11b0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42901 } }
I20260811 11:58:20.002244 23563 leader_election.cc:288] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2ed0ea19b39a461887cb746015b9d779 (127.6.143.130:43291), fa812b7ff43f40a7b6e2dc0c5fad925a (127.6.143.131:41443)
W20260811 11:58:20.003046 23080 leader_election.cc:341] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ed0ea19b39a461887cb746015b9d779 (127.6.143.130:43291): Not found: Tablet not found: 1ccbdfc56ca64bb2b2bb91d14877347b
W20260811 11:58:20.003211 23080 leader_election.cc:341] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fa812b7ff43f40a7b6e2dc0c5fad925a (127.6.143.131:41443): Not found: Tablet not found: 1ccbdfc56ca64bb2b2bb91d14877347b
I20260811 11:58:20.003278 23080 leader_election.cc:302] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e449f197ad66458097ce41adc60d11b0; no voters: 2ed0ea19b39a461887cb746015b9d779, fa812b7ff43f40a7b6e2dc0c5fad925a
I20260811 11:58:20.003463 23563 raft_consensus.cc:2726] T 1ccbdfc56ca64bb2b2bb91d14877347b P e449f197ad66458097ce41adc60d11b0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 11:58:20.249104 23564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:20.248584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.129:37809)
0811 11:58:20.248868 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:20.248956 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":75,"threads_started":1}
I20260811 11:58:20.391283 23565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:20.390894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.130:55303)
0811 11:58:20.391136 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:20.391190 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":73,"threads_started":1}
I20260811 11:58:20.570708 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 23027
I20260811 11:58:20.579583 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 23209
I20260811 11:58:20.583081 23566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:20.582559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39315 (local address 127.6.143.131:35167)
0811 11:58:20.582892 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:20.582979 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:39315: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":66,"threads_started":1}
I20260811 11:58:20.587869 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 23358
2026-08-11T11:58:20Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19404 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:58:20 dist-test-slave-zkpd krb5kdc[23570](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:58:20 dist-test-slave-zkpd krb5kdc[23570](info): set up 2 sockets
Aug 11 11:58:20 dist-test-slave-zkpd krb5kdc[23570](info): commencing operation
krb5kdc: starting...
W20260811 11:58:21.745482 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:58:21 dist-test-slave-zkpd krb5kdc[23570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:58:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:58:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
I20260811 11:58:21.878417 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:46307
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:33967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:46307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:58:21.990216 23583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:58:21.990451 23583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:58:21.990489 23583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:58:21.994318 23583 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 11:58:21.994364 23583 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 11:58:21.994379 23583 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 11:58:21.994392 23583 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:58:21.994410 23583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:58:21.994426 23583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:58:21.994449 23583 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:58:21.994468 23583 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:58:22.000077 23583 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:33967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:46307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:46307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:58:22.000996 23583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:58:22.001930 23583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:58:22.010373 23589 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:58:22.010540 23583 server_base.cc:1034] running on GCE node
W20260811 11:58:22.010462 23592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:22.010453 23588 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:58:22.011219 23583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:58:22.012324 23583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:58:22.013490 23583 hybrid_clock.cc:648] HybridClock initialized: now 1786449502013475 us; error 25 us; skew 500 ppm
Aug 11 11:58:22 dist-test-slave-zkpd krb5kdc[23570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449502, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:58:22.016366 23583 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:58:22.017244 23583 webserver.cc:460] Webserver started at http://127.6.143.190:33289/ using document root <none> and password file <none>
I20260811 11:58:22.017824 23583 fs_manager.cc:359] Metadata directory not provided
I20260811 11:58:22.017915 23583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:58:22.018150 23583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:58:22.020267 23583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "696c9ee4fe784fc39f4ab3e090399c7d"
format_stamp: "Formatted at 2026-08-11 11:58:22 on dist-test-slave-zkpd"
server_key: "d89de86cfcdf695666c0f4819218f007"
server_key_iv: "f50d9956f7d1755a31e50b0845fbc856"
server_key_version: "encryptionkey@0"
I20260811 11:58:22.020924 23583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "696c9ee4fe784fc39f4ab3e090399c7d"
format_stamp: "Formatted at 2026-08-11 11:58:22 on dist-test-slave-zkpd"
server_key: "d89de86cfcdf695666c0f4819218f007"
server_key_iv: "f50d9956f7d1755a31e50b0845fbc856"
server_key_version: "encryptionkey@0"
I20260811 11:58:22.024926 23583 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 11:58:22.027277 23599 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:22.028553 23583 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 11:58:22.028698 23583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "696c9ee4fe784fc39f4ab3e090399c7d"
format_stamp: "Formatted at 2026-08-11 11:58:22 on dist-test-slave-zkpd"
server_key: "d89de86cfcdf695666c0f4819218f007"
server_key_iv: "f50d9956f7d1755a31e50b0845fbc856"
server_key_version: "encryptionkey@0"
I20260811 11:58:22.028807 23583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:58:22.045043 23583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:58:22.047768 23583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:58:22.047999 23583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:58:22.055028 23583 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:46307
I20260811 11:58:22.055382 23655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:46307 every 8 connection(s)
I20260811 11:58:22.056032 23583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:58:22.058815 23656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:58:22.064843 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 23583
I20260811 11:58:22.064973 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:58:22.064934 23656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d: Bootstrap starting.
I20260811 11:58:22.065258 6718 external_mini_cluster.cc:1428] Setting key f2b7c246d6f5437c4ceadeabb832da2d
I20260811 11:58:22.067708 23656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d: Neither blocks nor log segments found. Creating new log.
I20260811 11:58:22.068619 23656 log.cc:824] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d: Log is configured to *not* fsync() on all Append() calls
I20260811 11:58:22.070870 23656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d: No bootstrap required, opened a new log
Aug 11 11:58:22 dist-test-slave-zkpd krb5kdc[23570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449501, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:58:22.075282 23656 raft_consensus.cc:348] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696c9ee4fe784fc39f4ab3e090399c7d" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46307 } }
I20260811 11:58:22.075539 23656 raft_consensus.cc:374] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:58:22.075609 23656 raft_consensus.cc:729] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 696c9ee4fe784fc39f4ab3e090399c7d, State: Initialized, Role: FOLLOWER
I20260811 11:58:22.076159 23656 consensus_queue.cc:260] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696c9ee4fe784fc39f4ab3e090399c7d" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46307 } }
I20260811 11:58:22.076346 23656 raft_consensus.cc:388] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:58:22.076423 23656 raft_consensus.cc:482] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:58:22.076541 23656 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:58:22.077677 23656 raft_consensus.cc:504] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696c9ee4fe784fc39f4ab3e090399c7d" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46307 } }
I20260811 11:58:22.078033 23656 leader_election.cc:302] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 696c9ee4fe784fc39f4ab3e090399c7d; no voters:
I20260811 11:58:22.078398 23656 leader_election.cc:288] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:58:22.078502 23662 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:58:22.078749 23662 raft_consensus.cc:686] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [term 1 LEADER]: Becoming Leader. State: Replica: 696c9ee4fe784fc39f4ab3e090399c7d, State: Running, Role: LEADER
I20260811 11:58:22.079066 23662 consensus_queue.cc:237] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696c9ee4fe784fc39f4ab3e090399c7d" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46307 } }
I20260811 11:58:22.080606 23656 sys_catalog.cc:564] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:58:22.080794 23663 sys_catalog.cc:455] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 696c9ee4fe784fc39f4ab3e090399c7d. Latest consensus state: current_term: 1 leader_uuid: "696c9ee4fe784fc39f4ab3e090399c7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696c9ee4fe784fc39f4ab3e090399c7d" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46307 } } }
I20260811 11:58:22.080906 23663 sys_catalog.cc:458] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [sys.catalog]: This master's current role is: LEADER
I20260811 11:58:22.081146 23664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "696c9ee4fe784fc39f4ab3e090399c7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696c9ee4fe784fc39f4ab3e090399c7d" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46307 } } }
I20260811 11:58:22.081255 23664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [sys.catalog]: This master's current role is: LEADER
I20260811 11:58:22.081400 23660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:22.066589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48544 (local address 127.6.143.190:46307)
0811 11:58:22.066993 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:22.067014 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:22.067059 (+ 45us) server_negotiation.cc:407] Connection header received
0811 11:58:22.067986 (+ 927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:22.068015 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:22.068439 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:22.068831 (+ 392us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:58:22.069713 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.071151 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.072040 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.072785 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.072941 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:22.075541 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:58:22.075549 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:58:22.075567 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:58:22.075600 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:58:22.077735 (+ 2135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:22.078186 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:22.078192 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:22.078206 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:22.078359 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:22.078680 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:22.078686 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:22.078691 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:22.079202 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:58:22.079387 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:22.080000 (+ 613us) server_negotiation.cc:299] Negotiation successful
0811 11:58:22.080305 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":96,"threads_started":1}
I20260811 11:58:22.081589 23667 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:58:22.084766 23667 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:58:22.090507 23667 catalog_manager.cc:1347] Generated new cluster ID: 0a31da5bc05f481ea0d7fe791ef87299
I20260811 11:58:22.090580 23667 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:58:22.105716 23667 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:58:22.106675 23667 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:58:22.112828 23667 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d: Generated new TSK 0
I20260811 11:58:22.113508 23667 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
I20260811 11:58:22.148813 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:0
--local_ip_for_outbound_sockets=127.6.143.129
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:46307
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:33967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:58:22.263002 23685 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:58:22.263250 23685 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:58:22.263291 23685 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:58:22.267271 23685 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:58:22.267329 23685 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:58:22.267390 23685 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:58:22.272409 23685 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:33967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:46307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23685
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:58:22.273319 23685 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:58:22.274302 23685 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:58:22.283919 23690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:22.283972 23691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:22.284078 23694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:58:22.284126 23685 server_base.cc:1034] running on GCE node
I20260811 11:58:22.284621 23685 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:58:22.285252 23685 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:58:22.286437 23685 hybrid_clock.cc:648] HybridClock initialized: now 1786449502286426 us; error 30 us; skew 500 ppm
Aug 11 11:58:22 dist-test-slave-zkpd krb5kdc[23570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449502, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:58:22.289188 23685 init.cc:377] Logged in from keytab as kudu/127.6.143.129@KRBTEST.COM (short username kudu)
I20260811 11:58:22.289965 23685 webserver.cc:460] Webserver started at http://127.6.143.129:40139/ using document root <none> and password file <none>
I20260811 11:58:22.290470 23685 fs_manager.cc:359] Metadata directory not provided
I20260811 11:58:22.290526 23685 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:58:22.290681 23685 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:58:22.292711 23685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/data/instance:
uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092"
format_stamp: "Formatted at 2026-08-11 11:58:22 on dist-test-slave-zkpd"
server_key: "e9768ccb17b7d068dfe7aea47e95640c"
server_key_iv: "79cb1c18e444eb829807aed0ee51e2be"
server_key_version: "encryptionkey@0"
I20260811 11:58:22.293227 23685 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance:
uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092"
format_stamp: "Formatted at 2026-08-11 11:58:22 on dist-test-slave-zkpd"
server_key: "e9768ccb17b7d068dfe7aea47e95640c"
server_key_iv: "79cb1c18e444eb829807aed0ee51e2be"
server_key_version: "encryptionkey@0"
I20260811 11:58:22.297369 23685 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 11:58:22.299666 23703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:22.300947 23685 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:58:22.301046 23685 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092"
format_stamp: "Formatted at 2026-08-11 11:58:22 on dist-test-slave-zkpd"
server_key: "e9768ccb17b7d068dfe7aea47e95640c"
server_key_iv: "79cb1c18e444eb829807aed0ee51e2be"
server_key_version: "encryptionkey@0"
I20260811 11:58:22.301132 23685 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:58:22.317423 23685 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:58:22.320402 23685 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:58:22.320565 23685 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:58:22.321110 23685 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:58:22.321940 23685 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:58:22.321991 23685 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:22.322033 23685 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:58:22.322057 23685 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:22.332438 23685 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:41993
I20260811 11:58:22.332468 23827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:41993 every 8 connection(s)
I20260811 11:58:22.333492 23685 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:58:22.335733 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 23685
I20260811 11:58:22.335891 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance
I20260811 11:58:22.336252 6718 external_mini_cluster.cc:1428] Setting key c35ca6e13d9dfa42f5cd848e54bf4e26
Aug 11 11:58:22 dist-test-slave-zkpd krb5kdc[23570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449502, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.143.130@KRBTEST.COM; defaulting to no policy
I20260811 11:58:22.349462 23660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:22.335411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:44473 (local address 127.6.143.190:46307)
0811 11:58:22.335614 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:22.335620 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:22.336912 (+ 1292us) server_negotiation.cc:407] Connection header received
0811 11:58:22.338075 (+ 1163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:22.338078 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:22.338124 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:22.338205 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:58:22.339823 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.340495 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.342187 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.342625 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.342758 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:22.345779 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:58:22.345783 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:58:22.345785 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:58:22.345809 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:58:22.347273 (+ 1464us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:22.347955 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:22.347958 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:22.347959 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:22.348073 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:22.348469 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:22.348472 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:22.348473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:22.348644 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:58:22.348722 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:22.349235 (+ 513us) server_negotiation.cc:299] Negotiation successful
0811 11:58:22.349338 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 11:58:22.350329 23830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:22.335888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.129:44473)
0811 11:58:22.336755 (+ 867us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:22.336799 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:22.337824 (+ 1025us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:22.338387 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:22.338399 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:22.338940 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:58:22.339650 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:22.339674 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.340637 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.340642 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:22.342034 (+ 1392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:22.342045 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.342782 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.342785 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:22.342939 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:22.343572 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:58:22.343600 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:58:22.345602 (+ 2002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:58:22.347422 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:22.347430 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:22.347450 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:22.347801 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:22.348170 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:22.348174 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:22.348178 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:22.348378 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:22.348739 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:58:22.348743 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:58:22.349022 (+ 279us) client_negotiation.cc:769] Sending connection context
0811 11:58:22.349183 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 11:58:22.349443 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":131,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.130@KRBTEST.COM" created.
I20260811 11:58:22.351785 23828 heartbeater.cc:347] Connected to a master server at 127.6.143.190:46307
I20260811 11:58:22.352061 23828 heartbeater.cc:464] Registering TS with master...
I20260811 11:58:22.352587 23828 heartbeater.cc:511] Master 127.6.143.190:46307 requested a full tablet report, sending...
I20260811 11:58:22.354233 23620 ts_manager.cc:194] Registered new tserver with Master: dc69aa9e3ea24eb6b8c4d3f92ad57092 (127.6.143.129:41993)
I20260811 11:58:22.355583 23620 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.129@KRBTEST.COM'} at 127.6.143.129:44473
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
I20260811 11:58:22.367067 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:0
--local_ip_for_outbound_sockets=127.6.143.130
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:46307
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:33967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:58:22.482568 23834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:58:22.482842 23834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:58:22.482918 23834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:58:22.486905 23834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:58:22.486986 23834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:58:22.487069 23834 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:58:22.492323 23834 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:33967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:46307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23834
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:58:22.493347 23834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:58:22.494417 23834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:58:22.503918 23843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:22.503947 23841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:58:22.504105 23834 server_base.cc:1034] running on GCE node
W20260811 11:58:22.503985 23840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:58:22.504664 23834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:58:22.505314 23834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:58:22.506486 23834 hybrid_clock.cc:648] HybridClock initialized: now 1786449502506454 us; error 42 us; skew 500 ppm
Aug 11 11:58:22 dist-test-slave-zkpd krb5kdc[23570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449502, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:58:22.513998 23834 init.cc:377] Logged in from keytab as kudu/127.6.143.130@KRBTEST.COM (short username kudu)
I20260811 11:58:22.514838 23834 webserver.cc:460] Webserver started at http://127.6.143.130:45449/ using document root <none> and password file <none>
I20260811 11:58:22.515422 23834 fs_manager.cc:359] Metadata directory not provided
I20260811 11:58:22.515516 23834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:58:22.515753 23834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:58:22.518191 23834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/data/instance:
uuid: "6e657e4123504248aad5c023b128faa1"
format_stamp: "Formatted at 2026-08-11 11:58:22 on dist-test-slave-zkpd"
server_key: "e24845d32925990f4b1eb464e62c85c3"
server_key_iv: "41cff7a9aaf750f98b47846238987975"
server_key_version: "encryptionkey@0"
I20260811 11:58:22.518767 23834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance:
uuid: "6e657e4123504248aad5c023b128faa1"
format_stamp: "Formatted at 2026-08-11 11:58:22 on dist-test-slave-zkpd"
server_key: "e24845d32925990f4b1eb464e62c85c3"
server_key_iv: "41cff7a9aaf750f98b47846238987975"
server_key_version: "encryptionkey@0"
I20260811 11:58:22.522464 23834 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260811 11:58:22.524941 23851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:22.526186 23834 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:58:22.526324 23834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "6e657e4123504248aad5c023b128faa1"
format_stamp: "Formatted at 2026-08-11 11:58:22 on dist-test-slave-zkpd"
server_key: "e24845d32925990f4b1eb464e62c85c3"
server_key_iv: "41cff7a9aaf750f98b47846238987975"
server_key_version: "encryptionkey@0"
I20260811 11:58:22.526443 23834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:58:22.561251 23834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:58:22.564369 23834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:58:22.564579 23834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:58:22.565199 23834 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:58:22.566057 23834 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:58:22.566130 23834 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:22.566206 23834 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:58:22.566250 23834 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:22.574838 23834 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:41053
I20260811 11:58:22.574849 23974 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:41053 every 8 connection(s)
I20260811 11:58:22.575816 23834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
I20260811 11:58:22.584030 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 23834
I20260811 11:58:22.584165 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance
I20260811 11:58:22.584506 6718 external_mini_cluster.cc:1428] Setting key c8626ff9030fb32561349e4ecc06afe9
Aug 11 11:58:22 dist-test-slave-zkpd krb5kdc[23570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449502, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:58:22.590528 23660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:22.577731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:55529 (local address 127.6.143.190:46307)
0811 11:58:22.577874 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:22.577878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:22.578753 (+ 875us) server_negotiation.cc:407] Connection header received
0811 11:58:22.580179 (+ 1426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:22.580182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:22.580241 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:22.580331 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:58:22.581598 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.582246 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.583470 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.583697 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.583810 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:22.586454 (+ 2644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:58:22.586460 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:58:22.586466 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:58:22.586499 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:58:22.588207 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:22.588869 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:22.588872 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:22.588873 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:22.588949 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:22.589343 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:22.589345 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:22.589346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:22.589514 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:58:22.589591 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:22.590224 (+ 633us) server_negotiation.cc:299] Negotiation successful
0811 11:58:22.590350 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 11:58:22.591356 23977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:22.578149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.130:55529)
0811 11:58:22.578605 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:22.578649 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:22.579955 (+ 1306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:22.580550 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:22.580557 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:22.580922 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:58:22.581418 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:22.581459 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.582370 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.582373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:22.583347 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:22.583357 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.583854 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.583866 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:22.584068 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:22.584727 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:58:22.584743 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:58:22.586279 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:58:22.588355 (+ 2076us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:22.588364 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:22.588383 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:22.588758 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:22.589044 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:22.589050 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:22.589053 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:22.589243 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:22.589621 (+ 378us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:58:22.589628 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:58:22.589974 (+ 346us) client_negotiation.cc:769] Sending connection context
0811 11:58:22.590176 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 11:58:22.590495 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":112,"threads_started":1}
I20260811 11:58:22.592543 23975 heartbeater.cc:347] Connected to a master server at 127.6.143.190:46307
I20260811 11:58:22.592795 23975 heartbeater.cc:464] Registering TS with master...
I20260811 11:58:22.593286 23975 heartbeater.cc:511] Master 127.6.143.190:46307 requested a full tablet report, sending...
I20260811 11:58:22.594419 23620 ts_manager.cc:194] Registered new tserver with Master: 6e657e4123504248aad5c023b128faa1 (127.6.143.130:41053)
I20260811 11:58:22.595244 23620 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.130@KRBTEST.COM'} at 127.6.143.130:55529
WARNING: no policy specified for kudu/127.6.143.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
I20260811 11:58:22.614147 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:0
--local_ip_for_outbound_sockets=127.6.143.131
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:46307
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:33967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:58:22.726652 23980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:58:22.726912 23980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:58:22.726953 23980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:58:22.731173 23980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:58:22.731232 23980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:58:22.731289 23980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:58:22.736426 23980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:33967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:46307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:58:22.737449 23980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:58:22.738494 23980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:58:22.748075 23986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:22.748070 23989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:58:22.748134 23980 server_base.cc:1034] running on GCE node
W20260811 11:58:22.748070 23985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:58:22.748771 23980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:58:22.749447 23980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:58:22.750638 23980 hybrid_clock.cc:648] HybridClock initialized: now 1786449502750622 us; error 26 us; skew 500 ppm
Aug 11 11:58:22 dist-test-slave-zkpd krb5kdc[23570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449502, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:58:22.753404 23980 init.cc:377] Logged in from keytab as kudu/127.6.143.131@KRBTEST.COM (short username kudu)
I20260811 11:58:22.754262 23980 webserver.cc:460] Webserver started at http://127.6.143.131:46379/ using document root <none> and password file <none>
I20260811 11:58:22.754833 23980 fs_manager.cc:359] Metadata directory not provided
I20260811 11:58:22.754946 23980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:58:22.755175 23980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:58:22.757344 23980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/data/instance:
uuid: "ab7a1e62d2584140a350aa043e080e21"
format_stamp: "Formatted at 2026-08-11 11:58:22 on dist-test-slave-zkpd"
server_key: "215383abe6819f4e96dab778758d3ecd"
server_key_iv: "c6a77d2642167cad1e94bd5f002da8ab"
server_key_version: "encryptionkey@0"
I20260811 11:58:22.757921 23980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance:
uuid: "ab7a1e62d2584140a350aa043e080e21"
format_stamp: "Formatted at 2026-08-11 11:58:22 on dist-test-slave-zkpd"
server_key: "215383abe6819f4e96dab778758d3ecd"
server_key_iv: "c6a77d2642167cad1e94bd5f002da8ab"
server_key_version: "encryptionkey@0"
I20260811 11:58:22.761619 23980 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 11:58:22.764017 23997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:22.765237 23980 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 11:58:22.765374 23980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "ab7a1e62d2584140a350aa043e080e21"
format_stamp: "Formatted at 2026-08-11 11:58:22 on dist-test-slave-zkpd"
server_key: "215383abe6819f4e96dab778758d3ecd"
server_key_iv: "c6a77d2642167cad1e94bd5f002da8ab"
server_key_version: "encryptionkey@0"
I20260811 11:58:22.765487 23980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:58:22.785385 23980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:58:22.788226 23980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:58:22.788427 23980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:58:22.789017 23980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:58:22.789851 23980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:58:22.789930 23980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:22.790024 23980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:58:22.790076 23980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:22.798767 23980 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:33253
I20260811 11:58:22.798791 24121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:33253 every 8 connection(s)
I20260811 11:58:22.799714 23980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:58:22.800993 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 23980
I20260811 11:58:22.801132 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance
I20260811 11:58:22.801477 6718 external_mini_cluster.cc:1428] Setting key 0b79a981ccabb564bcf09d525fa714e7
Aug 11 11:58:22 dist-test-slave-zkpd krb5kdc[23570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449502, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:58:22.813690 23660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:22.802001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:48551 (local address 127.6.143.190:46307)
0811 11:58:22.802169 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:22.802174 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:22.803280 (+ 1106us) server_negotiation.cc:407] Connection header received
0811 11:58:22.804379 (+ 1099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:22.804382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:22.804441 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:22.804545 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:58:22.805717 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.806246 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.807399 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.807694 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.807811 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:22.810272 (+ 2461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:58:22.810276 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:58:22.810279 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:58:22.810307 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:58:22.811553 (+ 1246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:22.812149 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:22.812152 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:22.812154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:22.812238 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:22.812587 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:22.812590 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:22.812592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:22.812774 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:58:22.812884 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:22.813385 (+ 501us) server_negotiation.cc:299] Negotiation successful
0811 11:58:22.813506 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 11:58:22.814667 24124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:22.802300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.131:48551)
0811 11:58:22.803147 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:22.803189 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:22.804170 (+ 981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:22.804728 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:22.804739 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:22.805107 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:58:22.805575 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:22.805587 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.806375 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.806378 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:22.807245 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:22.807255 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.807850 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.807854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:22.808072 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:22.808689 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:58:22.808706 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:58:22.810110 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:58:22.811680 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:22.811686 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:22.811700 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:22.812036 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:22.812332 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:22.812336 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:22.812338 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:22.812481 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:22.812887 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:58:22.812896 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:58:22.813133 (+ 237us) client_negotiation.cc:769] Sending connection context
0811 11:58:22.813316 (+ 183us) client_negotiation.cc:240] Negotiation successful
0811 11:58:22.813631 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":156,"threads_started":1}
I20260811 11:58:22.815790 24122 heartbeater.cc:347] Connected to a master server at 127.6.143.190:46307
I20260811 11:58:22.816082 24122 heartbeater.cc:464] Registering TS with master...
I20260811 11:58:22.816592 24122 heartbeater.cc:511] Master 127.6.143.190:46307 requested a full tablet report, sending...
I20260811 11:58:22.817618 23620 ts_manager.cc:194] Registered new tserver with Master: ab7a1e62d2584140a350aa043e080e21 (127.6.143.131:33253)
I20260811 11:58:22.818179 23620 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.131@KRBTEST.COM'} at 127.6.143.131:48551
I20260811 11:58:22.827327 6718 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:58:22.837034 23660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:22.829373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48556 (local address 127.6.143.190:46307)
0811 11:58:22.829547 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:22.829551 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:22.829692 (+ 141us) server_negotiation.cc:407] Connection header received
0811 11:58:22.829810 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:22.829812 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:22.829856 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:22.829926 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:58:22.830614 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.831146 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.832027 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.832235 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.832317 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:22.833767 (+ 1450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:58:22.833770 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:58:22.833772 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:58:22.833794 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:58:22.835264 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:22.835636 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:22.835638 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:22.835639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:22.835713 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:22.836110 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:22.836113 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:22.836114 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:22.836323 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:58:22.836493 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:22.836749 (+ 256us) server_negotiation.cc:299] Negotiation successful
0811 11:58:22.836868 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260811 11:58:22.839731 23620 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48556:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:58:22.842824 23620 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:58:22.857890 24135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:22.851413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:41993 (local address 127.0.0.1:35618)
0811 11:58:22.852519 (+ 1106us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:22.852534 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:22.852613 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:22.853330 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:22.853333 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:22.853350 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:58:22.853472 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:22.853479 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.854520 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.854525 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:22.856082 (+ 1557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:22.856092 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.857202 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.857206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:22.857368 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:22.857442 (+ 74us) client_negotiation.cc:769] Sending connection context
0811 11:58:22.857560 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 11:58:22.857718 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1036,"mutex_wait_us":78,"thread_start_us":98,"threads_started":1}
I20260811 11:58:22.858291 24139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:22.852180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47512 (local address 127.6.143.130:41053)
0811 11:58:22.852868 (+ 688us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:22.852877 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:22.852895 (+ 18us) server_negotiation.cc:407] Connection header received
0811 11:58:22.852956 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:22.852960 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:22.853141 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:22.853289 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:58:22.853727 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.854842 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.856342 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.857101 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.857199 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:22.857339 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:22.858037 (+ 698us) server_negotiation.cc:299] Negotiation successful
0811 11:58:22.858140 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":104,"threads_started":1}
I20260811 11:58:22.858491 24138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:22.852161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35618 (local address 127.6.143.129:41993)
0811 11:58:22.852805 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:22.852811 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:22.852831 (+ 20us) server_negotiation.cc:407] Connection header received
0811 11:58:22.852896 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:22.852901 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:22.853050 (+ 149us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:22.853175 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:58:22.853608 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.854396 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.856206 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.857050 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.857183 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:22.857339 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:22.857608 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 11:58:22.857713 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":91,"threads_started":1}
I20260811 11:58:22.857890 24133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:22.851713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:41053 (local address 127.0.0.1:47512)
0811 11:58:22.852616 (+ 903us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:22.852628 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:22.852693 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:22.853453 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:22.853458 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:22.853475 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:58:22.853589 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:22.853595 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.854987 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.854989 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:22.856225 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:22.856232 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.857202 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.857206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:22.857411 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:22.857441 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 11:58:22.857560 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 11:58:22.857718 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":856,"mutex_wait_us":116,"thread_start_us":97,"threads_started":1}
I20260811 11:58:22.859534 24137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:22.851351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:33253 (local address 127.0.0.1:39636)
0811 11:58:22.852215 (+ 864us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:22.852268 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:22.852412 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:22.853167 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:22.853173 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:22.853202 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:58:22.853340 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:22.853347 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.854893 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.854910 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:22.856213 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:22.856225 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.858345 (+ 2120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.858349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:22.858449 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:22.858475 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 11:58:22.858531 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 11:58:22.859393 (+ 862us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":690,"mutex_wait_us":26,"thread_start_us":71,"threads_started":1}
I20260811 11:58:22.860158 24136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:22.851497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39636 (local address 127.6.143.131:33253)
0811 11:58:22.852630 (+ 1133us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:22.852645 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:22.852675 (+ 30us) server_negotiation.cc:407] Connection header received
0811 11:58:22.852760 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:22.852765 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:22.852990 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:22.853163 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:58:22.853482 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.854732 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.857403 (+ 2671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.858196 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.859731 (+ 1535us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:22.859871 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:22.859968 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 11:58:22.860050 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1022,"thread_start_us":177,"threads_started":1}
I20260811 11:58:22.861778 23756 tablet_service.cc:1467] Processing CreateTablet for tablet a3ce7f29ce5d4766b44845f35ee271aa (DEFAULT_TABLE table=test-table [id=a5ba90004b8b4c22bc52ab41508dedb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:58:22.861778 23903 tablet_service.cc:1467] Processing CreateTablet for tablet a3ce7f29ce5d4766b44845f35ee271aa (DEFAULT_TABLE table=test-table [id=a5ba90004b8b4c22bc52ab41508dedb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:58:22.862950 23903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3ce7f29ce5d4766b44845f35ee271aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:58:22.862950 23756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3ce7f29ce5d4766b44845f35ee271aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:58:22.862613 24049 tablet_service.cc:1467] Processing CreateTablet for tablet a3ce7f29ce5d4766b44845f35ee271aa (DEFAULT_TABLE table=test-table [id=a5ba90004b8b4c22bc52ab41508dedb8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:58:22.863528 24049 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3ce7f29ce5d4766b44845f35ee271aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:58:22.869673 24140 tablet_bootstrap.cc:492] T a3ce7f29ce5d4766b44845f35ee271aa P ab7a1e62d2584140a350aa043e080e21: Bootstrap starting.
I20260811 11:58:22.871021 24141 tablet_bootstrap.cc:492] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092: Bootstrap starting.
I20260811 11:58:22.872293 24143 tablet_bootstrap.cc:492] T a3ce7f29ce5d4766b44845f35ee271aa P 6e657e4123504248aad5c023b128faa1: Bootstrap starting.
I20260811 11:58:22.872489 24140 tablet_bootstrap.cc:654] T a3ce7f29ce5d4766b44845f35ee271aa P ab7a1e62d2584140a350aa043e080e21: Neither blocks nor log segments found. Creating new log.
I20260811 11:58:22.873037 24141 tablet_bootstrap.cc:654] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092: Neither blocks nor log segments found. Creating new log.
I20260811 11:58:22.873404 24140 log.cc:824] T a3ce7f29ce5d4766b44845f35ee271aa P ab7a1e62d2584140a350aa043e080e21: Log is configured to *not* fsync() on all Append() calls
I20260811 11:58:22.873829 24141 log.cc:824] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092: Log is configured to *not* fsync() on all Append() calls
I20260811 11:58:22.874933 24143 tablet_bootstrap.cc:654] T a3ce7f29ce5d4766b44845f35ee271aa P 6e657e4123504248aad5c023b128faa1: Neither blocks nor log segments found. Creating new log.
I20260811 11:58:22.875607 24141 tablet_bootstrap.cc:492] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092: No bootstrap required, opened a new log
I20260811 11:58:22.875643 24140 tablet_bootstrap.cc:492] T a3ce7f29ce5d4766b44845f35ee271aa P ab7a1e62d2584140a350aa043e080e21: No bootstrap required, opened a new log
I20260811 11:58:22.875761 24143 log.cc:824] T a3ce7f29ce5d4766b44845f35ee271aa P 6e657e4123504248aad5c023b128faa1: Log is configured to *not* fsync() on all Append() calls
I20260811 11:58:22.875815 24141 ts_tablet_manager.cc:1397] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 11:58:22.875824 24140 ts_tablet_manager.cc:1397] T a3ce7f29ce5d4766b44845f35ee271aa P ab7a1e62d2584140a350aa043e080e21: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 11:58:22.877758 24143 tablet_bootstrap.cc:492] T a3ce7f29ce5d4766b44845f35ee271aa P 6e657e4123504248aad5c023b128faa1: No bootstrap required, opened a new log
I20260811 11:58:22.877948 24143 ts_tablet_manager.cc:1397] T a3ce7f29ce5d4766b44845f35ee271aa P 6e657e4123504248aad5c023b128faa1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 11:58:22.879730 24141 raft_consensus.cc:348] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 } } peers { permanent_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41993 } } peers { permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 } }
I20260811 11:58:22.879719 24140 raft_consensus.cc:348] T a3ce7f29ce5d4766b44845f35ee271aa P ab7a1e62d2584140a350aa043e080e21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 } } peers { permanent_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41993 } } peers { permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 } }
I20260811 11:58:22.880040 24140 raft_consensus.cc:374] T a3ce7f29ce5d4766b44845f35ee271aa P ab7a1e62d2584140a350aa043e080e21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:58:22.880043 24141 raft_consensus.cc:374] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:58:22.880105 24140 raft_consensus.cc:729] T a3ce7f29ce5d4766b44845f35ee271aa P ab7a1e62d2584140a350aa043e080e21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab7a1e62d2584140a350aa043e080e21, State: Initialized, Role: FOLLOWER
I20260811 11:58:22.880112 24141 raft_consensus.cc:729] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc69aa9e3ea24eb6b8c4d3f92ad57092, State: Initialized, Role: FOLLOWER
I20260811 11:58:22.880555 24141 consensus_queue.cc:260] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 } } peers { permanent_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41993 } } peers { permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 } }
I20260811 11:58:22.880584 24140 consensus_queue.cc:260] T a3ce7f29ce5d4766b44845f35ee271aa P ab7a1e62d2584140a350aa043e080e21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 } } peers { permanent_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41993 } } peers { permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 } }
I20260811 11:58:22.881611 24141 ts_tablet_manager.cc:1428] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 11:58:22.881626 24140 ts_tablet_manager.cc:1428] T a3ce7f29ce5d4766b44845f35ee271aa P ab7a1e62d2584140a350aa043e080e21: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 11:58:22.881999 23828 heartbeater.cc:503] Master 127.6.143.190:46307 was elected leader, sending a full tablet report...
I20260811 11:58:22.881861 24143 raft_consensus.cc:348] T a3ce7f29ce5d4766b44845f35ee271aa P 6e657e4123504248aad5c023b128faa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 } } peers { permanent_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41993 } } peers { permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 } }
I20260811 11:58:22.882193 24143 raft_consensus.cc:374] T a3ce7f29ce5d4766b44845f35ee271aa P 6e657e4123504248aad5c023b128faa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:58:22.882266 24143 raft_consensus.cc:729] T a3ce7f29ce5d4766b44845f35ee271aa P 6e657e4123504248aad5c023b128faa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e657e4123504248aad5c023b128faa1, State: Initialized, Role: FOLLOWER
I20260811 11:58:22.882405 24122 heartbeater.cc:503] Master 127.6.143.190:46307 was elected leader, sending a full tablet report...
I20260811 11:58:22.882752 24143 consensus_queue.cc:260] T a3ce7f29ce5d4766b44845f35ee271aa P 6e657e4123504248aad5c023b128faa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 } } peers { permanent_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41993 } } peers { permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 } }
I20260811 11:58:22.883567 23975 heartbeater.cc:503] Master 127.6.143.190:46307 was elected leader, sending a full tablet report...
I20260811 11:58:22.883653 24143 ts_tablet_manager.cc:1428] T a3ce7f29ce5d4766b44845f35ee271aa P 6e657e4123504248aad5c023b128faa1: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 11:58:22.972126 24150 raft_consensus.cc:482] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:58:22.972393 24150 raft_consensus.cc:504] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 } } peers { permanent_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41993 } } peers { permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 } }
I20260811 11:58:22.973578 24150 leader_election.cc:288] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab7a1e62d2584140a350aa043e080e21 (127.6.143.131:33253), 6e657e4123504248aad5c023b128faa1 (127.6.143.130:41053)
I20260811 11:58:22.977759 24139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:22.974007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:43241 (local address 127.6.143.130:41053)
0811 11:58:22.974148 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:22.974153 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:22.974447 (+ 294us) server_negotiation.cc:407] Connection header received
0811 11:58:22.974610 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:22.974613 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:22.974668 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:22.974757 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:58:22.975088 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.975577 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.976737 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.977136 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.977267 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:22.977327 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:22.977543 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 11:58:22.977617 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 11:58:22.977773 24136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:22.974070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:59153 (local address 127.6.143.131:33253)
0811 11:58:22.974213 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:22.974217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:22.974380 (+ 163us) server_negotiation.cc:407] Connection header received
0811 11:58:22.974566 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:22.974569 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:22.974619 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:22.974707 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:58:22.975019 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.975539 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.976751 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.977146 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.977279 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:22.977339 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:22.977570 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 11:58:22.977645 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 11:58:22.977833 24152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:22.973870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:33253 (local address 127.6.143.129:59153)
0811 11:58:22.974281 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:22.974293 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:22.974452 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:22.974726 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:22.974729 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:22.974745 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:58:22.974904 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:22.974910 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.975659 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.975664 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:22.976607 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:22.976617 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.977271 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.977275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:22.977441 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:22.977460 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 11:58:22.977551 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 11:58:22.977645 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"mutex_wait_us":82,"thread_start_us":127,"threads_started":1}
I20260811 11:58:22.978037 24153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:22.973870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:41053 (local address 127.6.143.129:43241)
0811 11:58:22.974326 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:22.974338 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:22.974481 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:22.974768 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:22.974771 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:22.974812 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:58:22.974966 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:22.974971 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.975695 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.975700 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:22.976606 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:22.976615 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:22.977265 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:22.977269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:22.977394 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:22.977436 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 11:58:22.977526 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 11:58:22.977609 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":399,"mutex_wait_us":115,"spinlock_wait_cycles":1920,"thread_start_us":126,"threads_started":1}
I20260811 11:58:22.978433 24071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3ce7f29ce5d4766b44845f35ee271aa" candidate_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab7a1e62d2584140a350aa043e080e21" is_pre_election: true
I20260811 11:58:22.978757 24071 raft_consensus.cc:2454] T a3ce7f29ce5d4766b44845f35ee271aa P ab7a1e62d2584140a350aa043e080e21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc69aa9e3ea24eb6b8c4d3f92ad57092 in term 0.
I20260811 11:58:22.978798 23924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3ce7f29ce5d4766b44845f35ee271aa" candidate_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e657e4123504248aad5c023b128faa1" is_pre_election: true
I20260811 11:58:22.979095 23924 raft_consensus.cc:2454] T a3ce7f29ce5d4766b44845f35ee271aa P 6e657e4123504248aad5c023b128faa1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc69aa9e3ea24eb6b8c4d3f92ad57092 in term 0.
I20260811 11:58:22.979295 23707 leader_election.cc:302] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ab7a1e62d2584140a350aa043e080e21, dc69aa9e3ea24eb6b8c4d3f92ad57092; no voters:
I20260811 11:58:22.979593 24150 raft_consensus.cc:2781] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:58:22.979677 24150 raft_consensus.cc:482] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:58:22.979709 24150 raft_consensus.cc:3037] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:58:22.980875 24150 raft_consensus.cc:504] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 } } peers { permanent_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41993 } } peers { permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 } }
I20260811 11:58:22.981243 24150 leader_election.cc:288] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092 [CANDIDATE]: Term 1 election: Requested vote from peers ab7a1e62d2584140a350aa043e080e21 (127.6.143.131:33253), 6e657e4123504248aad5c023b128faa1 (127.6.143.130:41053)
I20260811 11:58:22.981606 24071 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3ce7f29ce5d4766b44845f35ee271aa" candidate_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab7a1e62d2584140a350aa043e080e21"
I20260811 11:58:22.981669 23924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3ce7f29ce5d4766b44845f35ee271aa" candidate_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e657e4123504248aad5c023b128faa1"
I20260811 11:58:22.981756 24071 raft_consensus.cc:3037] T a3ce7f29ce5d4766b44845f35ee271aa P ab7a1e62d2584140a350aa043e080e21 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:58:22.981815 23924 raft_consensus.cc:3037] T a3ce7f29ce5d4766b44845f35ee271aa P 6e657e4123504248aad5c023b128faa1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:58:22.982856 23924 raft_consensus.cc:2454] T a3ce7f29ce5d4766b44845f35ee271aa P 6e657e4123504248aad5c023b128faa1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc69aa9e3ea24eb6b8c4d3f92ad57092 in term 1.
I20260811 11:58:22.982908 24071 raft_consensus.cc:2454] T a3ce7f29ce5d4766b44845f35ee271aa P ab7a1e62d2584140a350aa043e080e21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc69aa9e3ea24eb6b8c4d3f92ad57092 in term 1.
I20260811 11:58:22.983175 23704 leader_election.cc:302] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e657e4123504248aad5c023b128faa1, dc69aa9e3ea24eb6b8c4d3f92ad57092; no voters:
I20260811 11:58:22.983425 24150 raft_consensus.cc:2781] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:58:22.983672 24150 raft_consensus.cc:686] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 1 LEADER]: Becoming Leader. State: Replica: dc69aa9e3ea24eb6b8c4d3f92ad57092, State: Running, Role: LEADER
I20260811 11:58:22.984054 24150 consensus_queue.cc:237] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 } } peers { permanent_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41993 } } peers { permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 } }
I20260811 11:58:22.987223 23620 catalog_manager.cc:5495] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc69aa9e3ea24eb6b8c4d3f92ad57092 (127.6.143.129). New cstate: current_term: 1 leader_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41993 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 } health_report { overall_health: UNKNOWN } } }
I20260811 11:58:23.007838 24138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:23.004141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35620 (local address 127.6.143.129:41993)
0811 11:58:23.004275 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:23.004280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:23.004294 (+ 14us) server_negotiation.cc:407] Connection header received
0811 11:58:23.004472 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:23.004474 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:23.004529 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:23.004643 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:58:23.004887 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:23.005461 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:23.006357 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:23.006583 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:23.006696 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:23.007024 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:58:23.007441 (+ 417us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:58:23.007582 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:23.007626 (+ 44us) server_negotiation.cc:299] Negotiation successful
0811 11:58:23.007681 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 11:58:23.014194 23924 raft_consensus.cc:1261] T a3ce7f29ce5d4766b44845f35ee271aa P 6e657e4123504248aad5c023b128faa1 [term 1 FOLLOWER]: Refusing update from remote peer dc69aa9e3ea24eb6b8c4d3f92ad57092: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:58:23.014194 24071 raft_consensus.cc:1261] T a3ce7f29ce5d4766b44845f35ee271aa P ab7a1e62d2584140a350aa043e080e21 [term 1 FOLLOWER]: Refusing update from remote peer dc69aa9e3ea24eb6b8c4d3f92ad57092: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:58:23.015003 24154 consensus_queue.cc:1035] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:58:23.015265 24150 consensus_queue.cc:1035] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:58:23.023494 24160 mvcc.cc:205] Tried to move back new op lower bound from 7317297164338020352 to 7317297164225527808. Current Snapshot: MvccSnapshot[applied={T|T < 7317297164338020352}]
I20260811 11:58:23.023699 24159 mvcc.cc:205] Tried to move back new op lower bound from 7317297164338020352 to 7317297164225527808. Current Snapshot: MvccSnapshot[applied={T|T < 7317297164338020352}]
I20260811 11:58:23.032784 23618 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:58:23.032968 23618 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:58:23.035511 23618 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d: Sending DeleteTablet for 3 replicas of tablet a3ce7f29ce5d4766b44845f35ee271aa
I20260811 11:58:23.036300 24049 tablet_service.cc:1514] Processing DeleteTablet for tablet a3ce7f29ce5d4766b44845f35ee271aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:58:23 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:39636
I20260811 11:58:23.036296 23756 tablet_service.cc:1514] Processing DeleteTablet for tablet a3ce7f29ce5d4766b44845f35ee271aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:58:23 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:35618
I20260811 11:58:23.036500 23903 tablet_service.cc:1514] Processing DeleteTablet for tablet a3ce7f29ce5d4766b44845f35ee271aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:58:23 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:47512
I20260811 11:58:23.036900 24169 tablet_replica.cc:330] T a3ce7f29ce5d4766b44845f35ee271aa P ab7a1e62d2584140a350aa043e080e21: stopping tablet replica
I20260811 11:58:23.036948 24171 tablet_replica.cc:330] T a3ce7f29ce5d4766b44845f35ee271aa P 6e657e4123504248aad5c023b128faa1: stopping tablet replica
I20260811 11:58:23.037024 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 23685
I20260811 11:58:23.037257 24169 raft_consensus.cc:2229] T a3ce7f29ce5d4766b44845f35ee271aa P ab7a1e62d2584140a350aa043e080e21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:58:23.037361 24171 raft_consensus.cc:2229] T a3ce7f29ce5d4766b44845f35ee271aa P 6e657e4123504248aad5c023b128faa1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:58:23.038166 24169 raft_consensus.cc:2258] T a3ce7f29ce5d4766b44845f35ee271aa P ab7a1e62d2584140a350aa043e080e21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:58:23.038232 24171 raft_consensus.cc:2258] T a3ce7f29ce5d4766b44845f35ee271aa P 6e657e4123504248aad5c023b128faa1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:58:23.039685 24171 ts_tablet_manager.cc:1905] T a3ce7f29ce5d4766b44845f35ee271aa P 6e657e4123504248aad5c023b128faa1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:58:23.042372 24169 ts_tablet_manager.cc:1905] T a3ce7f29ce5d4766b44845f35ee271aa P ab7a1e62d2584140a350aa043e080e21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:58:23.042848 24171 ts_tablet_manager.cc:1918] T a3ce7f29ce5d4766b44845f35ee271aa P 6e657e4123504248aad5c023b128faa1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:58:23.043015 24171 log.cc:1196] T a3ce7f29ce5d4766b44845f35ee271aa P 6e657e4123504248aad5c023b128faa1: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/wal/wals/a3ce7f29ce5d4766b44845f35ee271aa
I20260811 11:58:23.043344 24171 ts_tablet_manager.cc:1939] T a3ce7f29ce5d4766b44845f35ee271aa P 6e657e4123504248aad5c023b128faa1: Deleting consensus metadata
I20260811 11:58:23.044785 23600 catalog_manager.cc:4841] TS 6e657e4123504248aad5c023b128faa1 (127.6.143.130:41053): tablet a3ce7f29ce5d4766b44845f35ee271aa (table test-table [id=a5ba90004b8b4c22bc52ab41508dedb8]) successfully deleted
I20260811 11:58:23.044992 24169 ts_tablet_manager.cc:1918] T a3ce7f29ce5d4766b44845f35ee271aa P ab7a1e62d2584140a350aa043e080e21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:58:23.045107 24169 log.cc:1196] T a3ce7f29ce5d4766b44845f35ee271aa P ab7a1e62d2584140a350aa043e080e21: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/wal/wals/a3ce7f29ce5d4766b44845f35ee271aa
I20260811 11:58:23.045420 24169 ts_tablet_manager.cc:1939] T a3ce7f29ce5d4766b44845f35ee271aa P ab7a1e62d2584140a350aa043e080e21: Deleting consensus metadata
I20260811 11:58:23.046413 23605 catalog_manager.cc:4841] TS ab7a1e62d2584140a350aa043e080e21 (127.6.143.131:33253): tablet a3ce7f29ce5d4766b44845f35ee271aa (table test-table [id=a5ba90004b8b4c22bc52ab41508dedb8]) successfully deleted
W20260811 11:58:23.048938 23601 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.143.129:41993 (error 108)
I20260811 11:58:23.049337 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 23834
I20260811 11:58:23.050369 24137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:23.049720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:41993 (local address 127.0.0.1:35632)
0811 11:58:23.050240 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:23.050296 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.129:41993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":436}
W20260811 11:58:23.050668 23601 catalog_manager.cc:4584] TS dc69aa9e3ea24eb6b8c4d3f92ad57092 (127.6.143.129:41993): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a3ce7f29ce5d4766b44845f35ee271aa: Network error: Client connection negotiation failed: client connection to 127.6.143.129:41993: connect: Connection refused (error 111)
I20260811 11:58:23.056795 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 23980
I20260811 11:58:23.064087 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 23583
I20260811 11:58:43.072402 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:46307
--webserver_interface=127.6.143.190
--webserver_port=33289
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:33967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:46307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:58:43.186421 24173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:58:43.186693 24173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:58:43.186770 24173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:58:43.190770 24173 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 11:58:43.190833 24173 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 11:58:43.190860 24173 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 11:58:43.190891 24173 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:58:43.190954 24173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:58:43.190997 24173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:58:43.191035 24173 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:58:43.191069 24173 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:58:43.196135 24173 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:33967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:46307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:46307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=33289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:58:43.197094 24173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:58:43.198079 24173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:58:43.212212 24180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:43.212200 24182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:43.212206 24179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:58:43.212407 24173 server_base.cc:1034] running on GCE node
I20260811 11:58:43.212999 24173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:58:43.214125 24173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:58:43.215307 24173 hybrid_clock.cc:648] HybridClock initialized: now 1786449523215289 us; error 33 us; skew 500 ppm
Aug 11 11:58:43 dist-test-slave-zkpd krb5kdc[23570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449523, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:58:43.218050 24173 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:58:43.218971 24173 webserver.cc:460] Webserver started at http://127.6.143.190:33289/ using document root <none> and password file <none>
I20260811 11:58:43.219573 24173 fs_manager.cc:359] Metadata directory not provided
I20260811 11:58:43.219677 24173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:58:43.223526 24173 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:58:43.225687 24189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:43.226881 24173 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 11:58:43.227030 24173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "696c9ee4fe784fc39f4ab3e090399c7d"
format_stamp: "Formatted at 2026-08-11 11:58:22 on dist-test-slave-zkpd"
server_key: "d89de86cfcdf695666c0f4819218f007"
server_key_iv: "f50d9956f7d1755a31e50b0845fbc856"
server_key_version: "encryptionkey@0"
I20260811 11:58:43.227644 24173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:58:43.246101 24173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:58:43.248962 24173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:58:43.249171 24173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:58:43.256206 24173 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:46307
I20260811 11:58:43.256497 24245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:46307 every 8 connection(s)
I20260811 11:58:43.257302 24173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:58:43.259709 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 24173
I20260811 11:58:43.260157 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:41993
--local_ip_for_outbound_sockets=127.6.143.129
--tserver_master_addrs=127.6.143.190:46307
--webserver_port=40139
--webserver_interface=127.6.143.129
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:33967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:58:43.261865 24246 sys_catalog.cc:263] Verifying existing consensus state
I20260811 11:58:43.263547 24246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d: Bootstrap starting.
I20260811 11:58:43.281530 24246 log.cc:824] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d: Log is configured to *not* fsync() on all Append() calls
I20260811 11:58:43.290103 24246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 11:58:43.290465 24246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d: Bootstrap complete.
I20260811 11:58:43.294674 24246 raft_consensus.cc:348] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696c9ee4fe784fc39f4ab3e090399c7d" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46307 } }
I20260811 11:58:43.295099 24246 raft_consensus.cc:729] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 696c9ee4fe784fc39f4ab3e090399c7d, State: Initialized, Role: FOLLOWER
I20260811 11:58:43.295619 24246 consensus_queue.cc:260] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696c9ee4fe784fc39f4ab3e090399c7d" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46307 } }
I20260811 11:58:43.295763 24246 raft_consensus.cc:388] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:58:43.295856 24246 raft_consensus.cc:482] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:58:43.296010 24246 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [term 1 FOLLOWER]: Advancing to term 2
I20260811 11:58:43.297861 24246 raft_consensus.cc:504] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696c9ee4fe784fc39f4ab3e090399c7d" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46307 } }
I20260811 11:58:43.298205 24246 leader_election.cc:302] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 696c9ee4fe784fc39f4ab3e090399c7d; no voters:
I20260811 11:58:43.298532 24246 leader_election.cc:288] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 11:58:43.298614 24250 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [term 2 FOLLOWER]: Leader election won for term 2
I20260811 11:58:43.298847 24250 raft_consensus.cc:686] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [term 2 LEADER]: Becoming Leader. State: Replica: 696c9ee4fe784fc39f4ab3e090399c7d, State: Running, Role: LEADER
I20260811 11:58:43.299191 24250 consensus_queue.cc:237] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696c9ee4fe784fc39f4ab3e090399c7d" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46307 } }
I20260811 11:58:43.299783 24246 sys_catalog.cc:564] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:58:43.300617 24251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "696c9ee4fe784fc39f4ab3e090399c7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696c9ee4fe784fc39f4ab3e090399c7d" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46307 } } }
I20260811 11:58:43.300758 24251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [sys.catalog]: This master's current role is: LEADER
I20260811 11:58:43.301460 24253 sys_catalog.cc:455] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 696c9ee4fe784fc39f4ab3e090399c7d. Latest consensus state: current_term: 2 leader_uuid: "696c9ee4fe784fc39f4ab3e090399c7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696c9ee4fe784fc39f4ab3e090399c7d" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46307 } } }
I20260811 11:58:43.301556 24253 sys_catalog.cc:458] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [sys.catalog]: This master's current role is: LEADER
I20260811 11:58:43.301767 24261 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:58:43.306550 24261 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:58:43.307356 24261 catalog_manager.cc:1259] Loaded cluster ID: 0a31da5bc05f481ea0d7fe791ef87299
I20260811 11:58:43.307428 24261 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:58:43.309990 24261 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:58:43.310732 24261 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d: Loaded TSK: 0
I20260811 11:58:43.324494 24261 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d: Generated new TSK 1
I20260811 11:58:43.324749 24261 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 11:58:43.377812 24248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:58:43.378093 24248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:58:43.378180 24248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:58:43.382176 24248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:58:43.382251 24248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:58:43.382333 24248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:58:43.387873 24248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:33967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:41993
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=40139
--tserver_master_addrs=127.6.143.190:46307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:58:43.388978 24248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:58:43.390002 24248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:58:43.400668 24278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:43.400810 24279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:43.400686 24281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:58:43.401114 24248 server_base.cc:1034] running on GCE node
I20260811 11:58:43.401585 24248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:58:43.402361 24248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:58:43.403571 24248 hybrid_clock.cc:648] HybridClock initialized: now 1786449523403530 us; error 44 us; skew 500 ppm
Aug 11 11:58:43 dist-test-slave-zkpd krb5kdc[23570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449523, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:58:43.406872 24248 init.cc:377] Logged in from keytab as kudu/127.6.143.129@KRBTEST.COM (short username kudu)
I20260811 11:58:43.407861 24248 webserver.cc:460] Webserver started at http://127.6.143.129:40139/ using document root <none> and password file <none>
I20260811 11:58:43.408488 24248 fs_manager.cc:359] Metadata directory not provided
I20260811 11:58:43.408565 24248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:58:43.412528 24248 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 11:58:43.414783 24290 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:43.416107 24248 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 11:58:43.416245 24248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092"
format_stamp: "Formatted at 2026-08-11 11:58:22 on dist-test-slave-zkpd"
server_key: "e9768ccb17b7d068dfe7aea47e95640c"
server_key_iv: "79cb1c18e444eb829807aed0ee51e2be"
server_key_version: "encryptionkey@0"
I20260811 11:58:43.416796 24248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:58:43.433902 24248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:58:43.436908 24248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:58:43.437101 24248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:58:43.437651 24248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:58:43.438724 24298 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 11:58:43.444032 24248 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 11:58:43.444134 24248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260811 11:58:43.444217 24248 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 11:58:43.445845 24248 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 11:58:43.445919 24248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 11:58:43.446281 24298 tablet_bootstrap.cc:492] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092: Bootstrap starting.
I20260811 11:58:43.456689 24248 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:41993
I20260811 11:58:43.456897 24417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:41993 every 8 connection(s)
I20260811 11:58:43.457708 24248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:58:43.467813 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 24248
I20260811 11:58:43.468344 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:41053
--local_ip_for_outbound_sockets=127.6.143.130
--tserver_master_addrs=127.6.143.190:46307
--webserver_port=45449
--webserver_interface=127.6.143.130
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:33967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:58:43.469077 24298 log.cc:824] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092: Log is configured to *not* fsync() on all Append() calls
Aug 11 11:58:43 dist-test-slave-zkpd krb5kdc[23570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449523, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:58:43.477085 24298 tablet_bootstrap.cc:492] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 11:58:43.477480 24298 tablet_bootstrap.cc:492] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092: Bootstrap complete.
I20260811 11:58:43.478911 24298 ts_tablet_manager.cc:1397] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092: Time spent bootstrapping tablet: real 0.033s user 0.027s sys 0.001s
I20260811 11:58:43.479442 24420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:43.459980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.129:48909)
0811 11:58:43.461748 (+ 1768us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:43.461807 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:43.462996 (+ 1189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:43.465116 (+ 2120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:43.465125 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:43.465665 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:58:43.466375 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:43.466400 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.468292 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.468297 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:43.469328 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:43.469339 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.470283 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.470286 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:43.470480 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:43.471212 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:58:43.471240 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:58:43.473152 (+ 1912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:58:43.475924 (+ 2772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:43.475929 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:43.475941 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:43.476263 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:43.476698 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:43.476701 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:43.476703 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:43.476895 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:43.477653 (+ 758us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:58:43.477659 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:58:43.477939 (+ 280us) client_negotiation.cc:769] Sending connection context
0811 11:58:43.478140 (+ 201us) client_negotiation.cc:240] Negotiation successful
0811 11:58:43.478490 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1509,"thread_start_us":143,"threads_started":1}
I20260811 11:58:43.479605 24421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:43.461747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:48909 (local address 127.6.143.190:46307)
0811 11:58:43.463435 (+ 1688us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:43.463457 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:43.463499 (+ 42us) server_negotiation.cc:407] Connection header received
0811 11:58:43.463805 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:43.463821 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:43.464248 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:43.464628 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:58:43.466608 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.468146 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.469455 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.470142 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.470319 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:43.473362 (+ 3043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:58:43.473371 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:58:43.473391 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:58:43.473429 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:58:43.475768 (+ 2339us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:43.476404 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:43.476413 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:43.476420 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:43.476587 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:43.477015 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:43.477021 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:43.477025 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:43.477535 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:58:43.477702 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:43.478324 (+ 622us) server_negotiation.cc:299] Negotiation successful
0811 11:58:43.478653 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1528,"thread_start_us":119,"threads_started":1}
I20260811 11:58:43.482227 24418 heartbeater.cc:347] Connected to a master server at 127.6.143.190:46307
I20260811 11:58:43.481900 24298 raft_consensus.cc:348] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 } } peers { permanent_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41993 } } peers { permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 } }
I20260811 11:58:43.482434 24298 raft_consensus.cc:729] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc69aa9e3ea24eb6b8c4d3f92ad57092, State: Initialized, Role: FOLLOWER
I20260811 11:58:43.482494 24418 heartbeater.cc:464] Registering TS with master...
I20260811 11:58:43.482918 24298 consensus_queue.cc:260] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 } } peers { permanent_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41993 } } peers { permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 } }
I20260811 11:58:43.483040 24418 heartbeater.cc:511] Master 127.6.143.190:46307 requested a full tablet report, sending...
I20260811 11:58:43.483788 24298 ts_tablet_manager.cc:1428] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 11:58:43.485599 24208 ts_manager.cc:194] Registered new tserver with Master: dc69aa9e3ea24eb6b8c4d3f92ad57092 (127.6.143.129:41993)
I20260811 11:58:43.489050 24208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.129@KRBTEST.COM'} at 127.6.143.129:48909
I20260811 11:58:43.491653 24418 heartbeater.cc:503] Master 127.6.143.190:46307 was elected leader, sending a full tablet report...
Aug 11 11:58:43 dist-test-slave-zkpd krb5kdc[23570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449523, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for kudu/127.6.143.129@KRBTEST.COM
I20260811 11:58:43.500689 24425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:43.488351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:41993 (local address 127.0.0.1:51042)
0811 11:58:43.488773 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:43.488820 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:43.488941 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:43.489557 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:43.489564 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:43.492019 (+ 2455us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:58:43.492166 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:43.492180 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.493108 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.493114 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:43.494095 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:43.494108 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.494721 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.494725 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:43.494841 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:43.495364 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:58:43.495381 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:58:43.496839 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:58:43.498752 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:43.498761 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:43.498767 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:43.499001 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:43.499309 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:43.499312 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:43.499314 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:43.499403 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:43.499941 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:58:43.499947 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:58:43.500074 (+ 127us) client_negotiation.cc:769] Sending connection context
0811 11:58:43.500264 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 11:58:43.500475 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":115,"threads_started":1}
I20260811 11:58:43.500744 24426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:43.488424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51042 (local address 127.6.143.129:41993)
0811 11:58:43.489090 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:43.489102 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:43.489128 (+ 26us) server_negotiation.cc:407] Connection header received
0811 11:58:43.489198 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:43.489203 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:43.489393 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:43.489526 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:58:43.492285 (+ 2759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.492990 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.494207 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.494596 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.494682 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:43.497009 (+ 2327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:58:43.497019 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:58:43.497036 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:58:43.497057 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:58:43.498620 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:43.499127 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:43.499133 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:43.499134 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:43.499211 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:43.499494 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:43.499499 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:43.499500 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:43.499788 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:58:43.499994 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:43.500359 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 11:58:43.500601 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":101,"threads_started":1}
I20260811 11:58:43.501956 24344 tablet_service.cc:1514] Processing DeleteTablet for tablet a3ce7f29ce5d4766b44845f35ee271aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:58:23 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:51042
I20260811 11:58:43.502794 24427 tablet_replica.cc:330] stopping tablet replica
I20260811 11:58:43.503099 24427 raft_consensus.cc:2229] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:58:43.503343 24427 raft_consensus.cc:2258] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:58:43.504403 24427 ts_tablet_manager.cc:1905] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:58:43.508996 24427 ts_tablet_manager.cc:1918] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:58:43.509097 24427 log.cc:1196] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/wal/wals/a3ce7f29ce5d4766b44845f35ee271aa
I20260811 11:58:43.509382 24427 ts_tablet_manager.cc:1939] T a3ce7f29ce5d4766b44845f35ee271aa P dc69aa9e3ea24eb6b8c4d3f92ad57092: Deleting consensus metadata
I20260811 11:58:43.511202 24191 catalog_manager.cc:4841] TS dc69aa9e3ea24eb6b8c4d3f92ad57092 (127.6.143.129:41993): tablet a3ce7f29ce5d4766b44845f35ee271aa (table test-table [id=a5ba90004b8b4c22bc52ab41508dedb8]) successfully deleted
W20260811 11:58:43.590170 24423 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:58:43.590404 24423 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:58:43.590483 24423 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:58:43.594444 24423 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:58:43.594502 24423 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:58:43.594614 24423 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:58:43.599715 24423 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:33967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:41053
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=45449
--tserver_master_addrs=127.6.143.190:46307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24423
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:58:43.600821 24423 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:58:43.601828 24423 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:58:43.611383 24436 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:43.611452 24432 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:43.611383 24434 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:58:43.611567 24423 server_base.cc:1034] running on GCE node
I20260811 11:58:43.612069 24423 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:58:43.612766 24423 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:58:43.613976 24423 hybrid_clock.cc:648] HybridClock initialized: now 1786449523613961 us; error 24 us; skew 500 ppm
Aug 11 11:58:43 dist-test-slave-zkpd krb5kdc[23570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449523, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:58:43.616727 24423 init.cc:377] Logged in from keytab as kudu/127.6.143.130@KRBTEST.COM (short username kudu)
I20260811 11:58:43.617604 24423 webserver.cc:460] Webserver started at http://127.6.143.130:45449/ using document root <none> and password file <none>
I20260811 11:58:43.618193 24423 fs_manager.cc:359] Metadata directory not provided
I20260811 11:58:43.618287 24423 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:58:43.622537 24423 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 11:58:43.624696 24444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:43.625897 24423 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:58:43.626032 24423 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "6e657e4123504248aad5c023b128faa1"
format_stamp: "Formatted at 2026-08-11 11:58:22 on dist-test-slave-zkpd"
server_key: "e24845d32925990f4b1eb464e62c85c3"
server_key_iv: "41cff7a9aaf750f98b47846238987975"
server_key_version: "encryptionkey@0"
I20260811 11:58:43.626634 24423 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:58:43.642742 24423 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:58:43.645592 24423 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:58:43.645795 24423 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:58:43.646399 24423 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:58:43.647272 24423 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:58:43.647341 24423 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:43.647408 24423 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:58:43.647447 24423 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:43.656231 24423 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:41053
I20260811 11:58:43.656296 24570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:41053 every 8 connection(s)
I20260811 11:58:43.657330 24423 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
Aug 11 11:58:43 dist-test-slave-zkpd krb5kdc[23570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449523, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:58:43.667098 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 24423
I20260811 11:58:43.667694 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:33253
--local_ip_for_outbound_sockets=127.6.143.131
--tserver_master_addrs=127.6.143.190:46307
--webserver_port=46379
--webserver_interface=127.6.143.131
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:33967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:58:43.671161 24421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:43.659184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:54113 (local address 127.6.143.190:46307)
0811 11:58:43.659363 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:43.659368 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:43.660487 (+ 1119us) server_negotiation.cc:407] Connection header received
0811 11:58:43.661333 (+ 846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:43.661335 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:43.661389 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:43.661475 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:58:43.662691 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.663227 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.664394 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.664623 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.664699 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:43.667180 (+ 2481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:58:43.667185 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:58:43.667189 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:58:43.667222 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:58:43.668961 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:43.669455 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:43.669462 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:43.669466 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:43.669571 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:43.670010 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:43.670012 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:43.670013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:43.670189 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:58:43.670267 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:43.670913 (+ 646us) server_negotiation.cc:299] Negotiation successful
0811 11:58:43.671026 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 11:58:43.672174 24574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:43.659614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.130:54113)
0811 11:58:43.660330 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:43.660382 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:43.661147 (+ 765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:43.661654 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:43.661661 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:43.662035 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:58:43.662554 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:43.662568 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.663335 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.663337 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:43.664284 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:43.664295 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.664748 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.664751 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:43.664885 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:43.665425 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:58:43.665441 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:58:43.666999 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:58:43.669088 (+ 2089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:43.669093 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:43.669105 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:43.669357 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:43.669695 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:43.669699 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:43.669702 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:43.669910 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:43.670292 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:58:43.670302 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:58:43.670635 (+ 333us) client_negotiation.cc:769] Sending connection context
0811 11:58:43.670871 (+ 236us) client_negotiation.cc:240] Negotiation successful
0811 11:58:43.671262 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":113,"threads_started":1}
I20260811 11:58:43.673525 24571 heartbeater.cc:347] Connected to a master server at 127.6.143.190:46307
I20260811 11:58:43.673827 24571 heartbeater.cc:464] Registering TS with master...
I20260811 11:58:43.674489 24571 heartbeater.cc:511] Master 127.6.143.190:46307 requested a full tablet report, sending...
I20260811 11:58:43.675693 24208 ts_manager.cc:194] Registered new tserver with Master: 6e657e4123504248aad5c023b128faa1 (127.6.143.130:41053)
I20260811 11:58:43.676297 24208 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.130@KRBTEST.COM'} at 127.6.143.130:54113
W20260811 11:58:43.782325 24575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:58:43.782616 24575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:58:43.782691 24575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:58:43.786701 24575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:58:43.786787 24575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:58:43.786875 24575 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:58:43.792198 24575 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:33967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:33253
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=46379
--tserver_master_addrs=127.6.143.190:46307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:58:43.793244 24575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:58:43.794267 24575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:58:43.804517 24583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:43.804517 24585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:58:43.804536 24582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:58:43.804754 24575 server_base.cc:1034] running on GCE node
I20260811 11:58:43.805181 24575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:58:43.805754 24575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:58:43.806922 24575 hybrid_clock.cc:648] HybridClock initialized: now 1786449523806907 us; error 27 us; skew 500 ppm
Aug 11 11:58:43 dist-test-slave-zkpd krb5kdc[23570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449523, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:58:43.809834 24575 init.cc:377] Logged in from keytab as kudu/127.6.143.131@KRBTEST.COM (short username kudu)
I20260811 11:58:43.810701 24575 webserver.cc:460] Webserver started at http://127.6.143.131:46379/ using document root <none> and password file <none>
I20260811 11:58:43.811270 24575 fs_manager.cc:359] Metadata directory not provided
I20260811 11:58:43.811373 24575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:58:43.815177 24575 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:58:43.817225 24592 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:43.818385 24575 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 11:58:43.818523 24575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "ab7a1e62d2584140a350aa043e080e21"
format_stamp: "Formatted at 2026-08-11 11:58:22 on dist-test-slave-zkpd"
server_key: "215383abe6819f4e96dab778758d3ecd"
server_key_iv: "c6a77d2642167cad1e94bd5f002da8ab"
server_key_version: "encryptionkey@0"
I20260811 11:58:43.819077 24575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:58:43.833633 24575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:58:43.836962 24575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:58:43.837158 24575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:58:43.837735 24575 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:58:43.838594 24575 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:58:43.838661 24575 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:43.838727 24575 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:58:43.838773 24575 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:58:43.847776 24575 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:33253
I20260811 11:58:43.847803 24716 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:33253 every 8 connection(s)
I20260811 11:58:43.848758 24575 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:58:43.855996 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 24575
Aug 11 11:58:43 dist-test-slave-zkpd krb5kdc[23570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449523, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:58:43.862353 24421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:43.850476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:38787 (local address 127.6.143.190:46307)
0811 11:58:43.850616 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:43.850621 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:43.851681 (+ 1060us) server_negotiation.cc:407] Connection header received
0811 11:58:43.852525 (+ 844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:43.852527 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:43.852575 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:43.852648 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:58:43.853741 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.854255 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.855400 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.855612 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.855687 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:43.858614 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:58:43.858617 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:58:43.858619 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:58:43.858642 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:58:43.860174 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:43.860796 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:43.860800 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:43.860806 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:43.860888 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:43.861227 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:43.861231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:43.861233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:43.861401 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:58:43.861506 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:43.862071 (+ 565us) server_negotiation.cc:299] Negotiation successful
0811 11:58:43.862185 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 11:58:43.863065 24719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:43.850908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.131:38787)
0811 11:58:43.851547 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:43.851587 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:43.852342 (+ 755us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:43.852808 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:43.852814 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:43.853145 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:58:43.853610 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:43.853623 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.854357 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.854360 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:43.855299 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:43.855309 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.855767 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.855773 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:43.855997 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:43.856717 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:58:43.856741 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:58:43.858459 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:58:43.860289 (+ 1830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:43.860295 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:43.860309 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:43.860676 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:43.860988 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:58:43.860991 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:58:43.860993 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:58:43.861131 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:58:43.861505 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:58:43.861512 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:58:43.861806 (+ 294us) client_negotiation.cc:769] Sending connection context
0811 11:58:43.862013 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 11:58:43.862308 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":107,"threads_started":1}
I20260811 11:58:43.864240 24717 heartbeater.cc:347] Connected to a master server at 127.6.143.190:46307
I20260811 11:58:43.864511 24717 heartbeater.cc:464] Registering TS with master...
I20260811 11:58:43.864882 24720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:43.856667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33696 (local address 127.6.143.190:46307)
0811 11:58:43.856906 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:43.856913 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:43.857100 (+ 187us) server_negotiation.cc:407] Connection header received
0811 11:58:43.857223 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:43.857225 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:43.857259 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:43.857331 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:58:43.858156 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.858869 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.859691 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.859965 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.860071 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:43.861336 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:58:43.861340 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:58:43.861342 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:58:43.861367 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:58:43.862941 (+ 1574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:43.863383 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:43.863388 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:43.863389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:43.863505 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:43.863967 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:43.863971 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:43.863972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:43.864167 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:58:43.864384 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:43.864588 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 11:58:43.864703 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"spinlock_wait_cycles":1792,"thread_start_us":75,"threads_started":1}
I20260811 11:58:43.865069 24717 heartbeater.cc:511] Master 127.6.143.190:46307 requested a full tablet report, sending...
I20260811 11:58:43.866195 24207 ts_manager.cc:194] Registered new tserver with Master: ab7a1e62d2584140a350aa043e080e21 (127.6.143.131:33253)
I20260811 11:58:43.866839 24207 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.131@KRBTEST.COM'} at 127.6.143.131:38787
I20260811 11:58:43.878129 6718 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:58:43.882285 24720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:43.878708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33698 (local address 127.6.143.190:46307)
0811 11:58:43.878849 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:43.878853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:43.879024 (+ 171us) server_negotiation.cc:407] Connection header received
0811 11:58:43.879207 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:43.879210 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:43.879312 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:43.879439 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:58:43.879671 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.880240 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.881088 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.881319 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.881433 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:43.881772 (+ 339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:58:43.881900 (+ 128us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 11:58:43.882163 (+ 263us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33698: token expired
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 11:58:43.882364 24720 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33698: token expired
W20260811 11:58:43.882675 24126 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.6.143.190:46307)
I20260811 11:58:43.889353 24720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:43.883015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33700 (local address 127.6.143.190:46307)
0811 11:58:43.883117 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:43.883121 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:43.883137 (+ 16us) server_negotiation.cc:407] Connection header received
0811 11:58:43.883216 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:43.883218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:43.883249 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:43.883311 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:58:43.883858 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.884347 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.885079 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.885268 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.885338 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:43.886529 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:58:43.886532 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:58:43.886534 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:58:43.886567 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:58:43.888075 (+ 1508us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:43.888429 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:43.888432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:43.888433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:43.888504 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:58:43.888725 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:58:43.888727 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:58:43.888728 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:58:43.888875 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:58:43.888975 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:43.889163 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 11:58:43.889248 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 11:58:43.891521 24207 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33700:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:58:43.894829 24207 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:58:43.903280 24344 tablet_service.cc:1467] Processing CreateTablet for tablet 7cb231f466384a7fa4a57ed88980b87f (DEFAULT_TABLE table=test-table [id=82cc6e031b7447c2bea21ec38aa067f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:58:43.904004 24344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cb231f466384a7fa4a57ed88980b87f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:58:43.906937 24298 tablet_bootstrap.cc:492] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092: Bootstrap starting.
I20260811 11:58:43.907068 24425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:43.901713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:41053 (local address 127.0.0.1:59276)
0811 11:58:43.901849 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:43.901867 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:43.901954 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:43.902933 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:43.902936 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:43.902955 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:58:43.903053 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:43.903059 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.904691 (+ 1632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.904695 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:43.905550 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:43.905560 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.906633 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.906636 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:43.906740 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:43.906781 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 11:58:43.906866 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 11:58:43.906946 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 11:58:43.907119 24726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:43.901926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59276 (local address 127.6.143.130:41053)
0811 11:58:43.902390 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:43.902402 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:43.902427 (+ 25us) server_negotiation.cc:407] Connection header received
0811 11:58:43.902512 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:43.902517 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:43.902761 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:43.903093 (+ 332us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:58:43.903277 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.904562 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.905672 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.906510 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.906625 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:43.906774 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:43.906899 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 11:58:43.906988 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":130,"threads_started":1}
I20260811 11:58:43.908047 24298 tablet_bootstrap.cc:654] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092: Neither blocks nor log segments found. Creating new log.
I20260811 11:58:43.908059 24728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:43.902014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:33253 (local address 127.0.0.1:50200)
0811 11:58:43.902281 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:43.902295 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:43.902394 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:43.903522 (+ 1128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:43.903526 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:43.903544 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:58:43.903653 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:43.903660 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.904951 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.904955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:43.906033 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:43.906044 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.907477 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.907481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:43.907580 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:43.907603 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 11:58:43.907654 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 11:58:43.907718 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":120,"threads_started":1}
I20260811 11:58:43.908829 24729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:43.902518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50200 (local address 127.6.143.131:33253)
0811 11:58:43.902848 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:43.902859 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:43.902897 (+ 38us) server_negotiation.cc:407] Connection header received
0811 11:58:43.902973 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:43.902978 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:43.903340 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:43.903788 (+ 448us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:58:43.903976 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.904825 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.906179 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.907100 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.908334 (+ 1234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:43.908509 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:43.908569 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 11:58:43.908665 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":141,"threads_started":1}
I20260811 11:58:43.910178 24499 tablet_service.cc:1467] Processing CreateTablet for tablet 7cb231f466384a7fa4a57ed88980b87f (DEFAULT_TABLE table=test-table [id=82cc6e031b7447c2bea21ec38aa067f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:58:43.911182 24499 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cb231f466384a7fa4a57ed88980b87f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:58:43.911455 24647 tablet_service.cc:1467] Processing CreateTablet for tablet 7cb231f466384a7fa4a57ed88980b87f (DEFAULT_TABLE table=test-table [id=82cc6e031b7447c2bea21ec38aa067f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:58:43.912307 24647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7cb231f466384a7fa4a57ed88980b87f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:58:43.916724 24298 tablet_bootstrap.cc:492] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092: No bootstrap required, opened a new log
I20260811 11:58:43.916801 24298 ts_tablet_manager.cc:1397] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260811 11:58:43.917238 24298 raft_consensus.cc:348] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 } } peers { permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 } } peers { permanent_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41993 } }
I20260811 11:58:43.917336 24298 raft_consensus.cc:374] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:58:43.917363 24298 raft_consensus.cc:729] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc69aa9e3ea24eb6b8c4d3f92ad57092, State: Initialized, Role: FOLLOWER
I20260811 11:58:43.917518 24298 consensus_queue.cc:260] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 } } peers { permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 } } peers { permanent_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41993 } }
I20260811 11:58:43.917780 24298 ts_tablet_manager.cc:1428] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 11:58:43.919126 24732 tablet_bootstrap.cc:492] T 7cb231f466384a7fa4a57ed88980b87f P 6e657e4123504248aad5c023b128faa1: Bootstrap starting.
I20260811 11:58:43.921062 24733 tablet_bootstrap.cc:492] T 7cb231f466384a7fa4a57ed88980b87f P ab7a1e62d2584140a350aa043e080e21: Bootstrap starting.
I20260811 11:58:43.921939 24732 tablet_bootstrap.cc:654] T 7cb231f466384a7fa4a57ed88980b87f P 6e657e4123504248aad5c023b128faa1: Neither blocks nor log segments found. Creating new log.
I20260811 11:58:43.922817 24732 log.cc:824] T 7cb231f466384a7fa4a57ed88980b87f P 6e657e4123504248aad5c023b128faa1: Log is configured to *not* fsync() on all Append() calls
I20260811 11:58:43.923345 24733 tablet_bootstrap.cc:654] T 7cb231f466384a7fa4a57ed88980b87f P ab7a1e62d2584140a350aa043e080e21: Neither blocks nor log segments found. Creating new log.
I20260811 11:58:43.924141 24733 log.cc:824] T 7cb231f466384a7fa4a57ed88980b87f P ab7a1e62d2584140a350aa043e080e21: Log is configured to *not* fsync() on all Append() calls
I20260811 11:58:43.924880 24732 tablet_bootstrap.cc:492] T 7cb231f466384a7fa4a57ed88980b87f P 6e657e4123504248aad5c023b128faa1: No bootstrap required, opened a new log
I20260811 11:58:43.925082 24732 ts_tablet_manager.cc:1397] T 7cb231f466384a7fa4a57ed88980b87f P 6e657e4123504248aad5c023b128faa1: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 11:58:43.925869 24733 tablet_bootstrap.cc:492] T 7cb231f466384a7fa4a57ed88980b87f P ab7a1e62d2584140a350aa043e080e21: No bootstrap required, opened a new log
I20260811 11:58:43.926056 24733 ts_tablet_manager.cc:1397] T 7cb231f466384a7fa4a57ed88980b87f P ab7a1e62d2584140a350aa043e080e21: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 11:58:43.928117 24732 raft_consensus.cc:348] T 7cb231f466384a7fa4a57ed88980b87f P 6e657e4123504248aad5c023b128faa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 } } peers { permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 } } peers { permanent_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41993 } }
I20260811 11:58:43.928365 24732 raft_consensus.cc:374] T 7cb231f466384a7fa4a57ed88980b87f P 6e657e4123504248aad5c023b128faa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:58:43.928432 24732 raft_consensus.cc:729] T 7cb231f466384a7fa4a57ed88980b87f P 6e657e4123504248aad5c023b128faa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e657e4123504248aad5c023b128faa1, State: Initialized, Role: FOLLOWER
I20260811 11:58:43.928884 24732 consensus_queue.cc:260] T 7cb231f466384a7fa4a57ed88980b87f P 6e657e4123504248aad5c023b128faa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 } } peers { permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 } } peers { permanent_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41993 } }
I20260811 11:58:43.929143 24733 raft_consensus.cc:348] T 7cb231f466384a7fa4a57ed88980b87f P ab7a1e62d2584140a350aa043e080e21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 } } peers { permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 } } peers { permanent_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41993 } }
I20260811 11:58:43.929419 24733 raft_consensus.cc:374] T 7cb231f466384a7fa4a57ed88980b87f P ab7a1e62d2584140a350aa043e080e21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:58:43.929481 24733 raft_consensus.cc:729] T 7cb231f466384a7fa4a57ed88980b87f P ab7a1e62d2584140a350aa043e080e21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab7a1e62d2584140a350aa043e080e21, State: Initialized, Role: FOLLOWER
I20260811 11:58:43.929579 24571 heartbeater.cc:503] Master 127.6.143.190:46307 was elected leader, sending a full tablet report...
I20260811 11:58:43.929842 24732 ts_tablet_manager.cc:1428] T 7cb231f466384a7fa4a57ed88980b87f P 6e657e4123504248aad5c023b128faa1: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 11:58:43.929875 24733 consensus_queue.cc:260] T 7cb231f466384a7fa4a57ed88980b87f P ab7a1e62d2584140a350aa043e080e21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 } } peers { permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 } } peers { permanent_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41993 } }
I20260811 11:58:43.930510 24717 heartbeater.cc:503] Master 127.6.143.190:46307 was elected leader, sending a full tablet report...
I20260811 11:58:43.930936 24733 ts_tablet_manager.cc:1428] T 7cb231f466384a7fa4a57ed88980b87f P ab7a1e62d2584140a350aa043e080e21: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260811 11:58:43.961920 24419 tablet.cc:2369] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:58:43.967392 24424 raft_consensus.cc:482] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:58:43.967574 24424 raft_consensus.cc:504] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 } } peers { permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 } } peers { permanent_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41993 } }
I20260811 11:58:43.968652 24424 leader_election.cc:288] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e657e4123504248aad5c023b128faa1 (127.6.143.130:41053), ab7a1e62d2584140a350aa043e080e21 (127.6.143.131:33253)
I20260811 11:58:43.973475 24729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:43.969064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:46679 (local address 127.6.143.131:33253)
0811 11:58:43.969245 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:43.969249 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:43.969377 (+ 128us) server_negotiation.cc:407] Connection header received
0811 11:58:43.969491 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:43.969494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:43.969542 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:43.969610 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:58:43.969882 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.970607 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.972109 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.972728 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.972855 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:43.972907 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:43.973285 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 11:58:43.973352 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 11:58:43.973475 24726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:43.969003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:38387 (local address 127.6.143.130:41053)
0811 11:58:43.969129 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:43.969136 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:43.969155 (+ 19us) server_negotiation.cc:407] Connection header received
0811 11:58:43.969268 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:43.969272 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:43.969322 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:43.969425 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:58:43.969733 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.970500 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.972110 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.972728 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.972854 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:43.972921 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:43.973285 (+ 364us) server_negotiation.cc:299] Negotiation successful
0811 11:58:43.973352 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 11:58:43.973496 24420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:43.968865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:41053 (local address 127.6.143.129:38387)
0811 11:58:43.969018 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:43.969034 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:43.969140 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:43.969452 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:43.969455 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:43.969474 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:58:43.969614 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:43.969621 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.970621 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.970623 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:43.971988 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:43.972001 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.972846 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.972850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:43.973080 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:43.973173 (+ 93us) client_negotiation.cc:769] Sending connection context
0811 11:58:43.973256 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 11:58:43.973340 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":158}
I20260811 11:58:43.973520 24738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:43.968892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:33253 (local address 127.6.143.129:46679)
0811 11:58:43.969284 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:43.969296 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:43.969379 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:43.969643 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:43.969649 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:43.969670 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:58:43.969786 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:43.969791 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.970701 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.970703 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:43.972001 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:43.972008 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.972880 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.972883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:43.973077 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:43.973175 (+ 98us) client_negotiation.cc:769] Sending connection context
0811 11:58:43.973256 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 11:58:43.973340 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"mutex_wait_us":137,"thread_start_us":116,"threads_started":1}
I20260811 11:58:43.974099 24737 raft_consensus.cc:482] T 7cb231f466384a7fa4a57ed88980b87f P ab7a1e62d2584140a350aa043e080e21 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:58:43.974190 24521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cb231f466384a7fa4a57ed88980b87f" candidate_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e657e4123504248aad5c023b128faa1" is_pre_election: true
I20260811 11:58:43.974228 24669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cb231f466384a7fa4a57ed88980b87f" candidate_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab7a1e62d2584140a350aa043e080e21" is_pre_election: true
I20260811 11:58:43.974272 24737 raft_consensus.cc:504] T 7cb231f466384a7fa4a57ed88980b87f P ab7a1e62d2584140a350aa043e080e21 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 } } peers { permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 } } peers { permanent_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41993 } }
I20260811 11:58:43.974493 24521 raft_consensus.cc:2454] T 7cb231f466384a7fa4a57ed88980b87f P 6e657e4123504248aad5c023b128faa1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc69aa9e3ea24eb6b8c4d3f92ad57092 in term 0.
I20260811 11:58:43.974686 24669 raft_consensus.cc:2454] T 7cb231f466384a7fa4a57ed88980b87f P ab7a1e62d2584140a350aa043e080e21 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc69aa9e3ea24eb6b8c4d3f92ad57092 in term 0.
I20260811 11:58:43.974987 24292 leader_election.cc:302] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e657e4123504248aad5c023b128faa1, dc69aa9e3ea24eb6b8c4d3f92ad57092; no voters:
I20260811 11:58:43.975266 24424 raft_consensus.cc:2781] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:58:43.975353 24424 raft_consensus.cc:482] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:58:43.975418 24424 raft_consensus.cc:3037] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:58:43.975792 24737 leader_election.cc:288] T 7cb231f466384a7fa4a57ed88980b87f P ab7a1e62d2584140a350aa043e080e21 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e657e4123504248aad5c023b128faa1 (127.6.143.130:41053), dc69aa9e3ea24eb6b8c4d3f92ad57092 (127.6.143.129:41993)
I20260811 11:58:43.976982 24424 raft_consensus.cc:504] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 } } peers { permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 } } peers { permanent_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41993 } }
I20260811 11:58:43.977360 24424 leader_election.cc:288] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [CANDIDATE]: Term 1 election: Requested vote from peers 6e657e4123504248aad5c023b128faa1 (127.6.143.130:41053), ab7a1e62d2584140a350aa043e080e21 (127.6.143.131:33253)
I20260811 11:58:43.977780 24521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cb231f466384a7fa4a57ed88980b87f" candidate_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e657e4123504248aad5c023b128faa1"
I20260811 11:58:43.977926 24521 raft_consensus.cc:3037] T 7cb231f466384a7fa4a57ed88980b87f P 6e657e4123504248aad5c023b128faa1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:58:43.978340 24669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cb231f466384a7fa4a57ed88980b87f" candidate_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab7a1e62d2584140a350aa043e080e21"
I20260811 11:58:43.978461 24669 raft_consensus.cc:3037] T 7cb231f466384a7fa4a57ed88980b87f P ab7a1e62d2584140a350aa043e080e21 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:58:43.979430 24521 raft_consensus.cc:2454] T 7cb231f466384a7fa4a57ed88980b87f P 6e657e4123504248aad5c023b128faa1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc69aa9e3ea24eb6b8c4d3f92ad57092 in term 1.
I20260811 11:58:43.979679 24669 raft_consensus.cc:2454] T 7cb231f466384a7fa4a57ed88980b87f P ab7a1e62d2584140a350aa043e080e21 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc69aa9e3ea24eb6b8c4d3f92ad57092 in term 1.
I20260811 11:58:43.979785 24292 leader_election.cc:302] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6e657e4123504248aad5c023b128faa1, dc69aa9e3ea24eb6b8c4d3f92ad57092; no voters:
I20260811 11:58:43.980252 24719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:43.975579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:41053 (local address 127.6.143.131:51929)
0811 11:58:43.975694 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:43.975708 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:43.975784 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:43.976092 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:43.976098 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:43.976123 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:58:43.976233 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:43.976239 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.976994 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.976999 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:43.978148 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:43.978158 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.979842 (+ 1684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.979846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:43.979987 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:43.980014 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 11:58:43.980067 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 11:58:43.980132 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260811 11:58:43.980690 24726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:43.975657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:51929 (local address 127.6.143.130:41053)
0811 11:58:43.975757 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:43.975760 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:43.975776 (+ 16us) server_negotiation.cc:407] Connection header received
0811 11:58:43.975954 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:43.975958 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:43.975999 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:43.976068 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:58:43.976376 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.976882 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.978881 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.979644 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.980437 (+ 793us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:43.980496 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:43.980532 (+ 36us) server_negotiation.cc:299] Negotiation successful
0811 11:58:43.980587 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 11:58:43.980711 24424 raft_consensus.cc:2781] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:58:43.980927 24424 raft_consensus.cc:686] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 1 LEADER]: Becoming Leader. State: Replica: dc69aa9e3ea24eb6b8c4d3f92ad57092, State: Running, Role: LEADER
I20260811 11:58:43.980965 24426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:43.976333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:41387 (local address 127.6.143.129:41993)
0811 11:58:43.976437 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:43.976441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:43.976453 (+ 12us) server_negotiation.cc:407] Connection header received
0811 11:58:43.976589 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:43.976592 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:43.976633 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:43.976677 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:58:43.977071 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.977835 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.979616 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.980436 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.980543 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:43.980650 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:43.980750 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 11:58:43.980848 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 11:58:43.981105 24739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:43.976020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:41993 (local address 127.6.143.131:41387)
0811 11:58:43.976358 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:43.976371 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:58:43.976465 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:58:43.976804 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:58:43.976809 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:58:43.976832 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:58:43.976961 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:43.976969 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.978267 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.978270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:43.979461 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:58:43.979473 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:58:43.980541 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:58:43.980544 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:58:43.980620 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:43.980639 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 11:58:43.980730 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 11:58:43.980966 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":132,"threads_started":1}
I20260811 11:58:43.981232 24424 consensus_queue.cc:237] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 } } peers { permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 } } peers { permanent_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41993 } }
I20260811 11:58:43.981367 24521 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cb231f466384a7fa4a57ed88980b87f" candidate_uuid: "ab7a1e62d2584140a350aa043e080e21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e657e4123504248aad5c023b128faa1" is_pre_election: true
I20260811 11:58:43.981515 24521 raft_consensus.cc:2379] T 7cb231f466384a7fa4a57ed88980b87f P 6e657e4123504248aad5c023b128faa1 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ab7a1e62d2584140a350aa043e080e21 in current term 1: Already voted for candidate dc69aa9e3ea24eb6b8c4d3f92ad57092 in this term.
I20260811 11:58:43.981775 24367 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7cb231f466384a7fa4a57ed88980b87f" candidate_uuid: "ab7a1e62d2584140a350aa043e080e21" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" is_pre_election: true
I20260811 11:58:43.982491 24594 leader_election.cc:302] T 7cb231f466384a7fa4a57ed88980b87f P ab7a1e62d2584140a350aa043e080e21 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ab7a1e62d2584140a350aa043e080e21; no voters: 6e657e4123504248aad5c023b128faa1, dc69aa9e3ea24eb6b8c4d3f92ad57092
I20260811 11:58:43.982954 24737 raft_consensus.cc:2726] T 7cb231f466384a7fa4a57ed88980b87f P ab7a1e62d2584140a350aa043e080e21 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 11:58:43.984306 24208 catalog_manager.cc:5495] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 reported cstate change: term changed from 0 to 1, leader changed from <none> to dc69aa9e3ea24eb6b8c4d3f92ad57092 (127.6.143.129). New cstate: current_term: 1 leader_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41993 } health_report { overall_health: HEALTHY } } }
I20260811 11:58:44.005578 24426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:44.002045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51054 (local address 127.6.143.129:41993)
0811 11:58:44.002173 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 11:58:44.002179 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:58:44.002195 (+ 16us) server_negotiation.cc:407] Connection header received
0811 11:58:44.002379 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:58:44.002382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:58:44.002442 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:58:44.002550 (+ 108us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:58:44.002792 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:44.003289 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:44.004186 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:58:44.004418 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:58:44.004521 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:58:44.004798 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:58:44.005205 (+ 407us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:58:44.005331 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 11:58:44.005394 (+ 63us) server_negotiation.cc:299] Negotiation successful
0811 11:58:44.005445 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 11:58:44.009421 24521 raft_consensus.cc:1261] T 7cb231f466384a7fa4a57ed88980b87f P 6e657e4123504248aad5c023b128faa1 [term 1 FOLLOWER]: Refusing update from remote peer dc69aa9e3ea24eb6b8c4d3f92ad57092: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:58:44.009470 24669 raft_consensus.cc:1261] T 7cb231f466384a7fa4a57ed88980b87f P ab7a1e62d2584140a350aa043e080e21 [term 1 FOLLOWER]: Refusing update from remote peer dc69aa9e3ea24eb6b8c4d3f92ad57092: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:58:44.010037 24740 consensus_queue.cc:1035] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:58:44.010207 24424 consensus_queue.cc:1035] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:58:44.025949 24207 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:58:44.026095 24207 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33700:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:58:44.028409 24207 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d: Sending DeleteTablet for 3 replicas of tablet 7cb231f466384a7fa4a57ed88980b87f
I20260811 11:58:44.029105 24344 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cb231f466384a7fa4a57ed88980b87f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:58:44 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:51042
I20260811 11:58:44.029130 24499 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cb231f466384a7fa4a57ed88980b87f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:58:44 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:59276
I20260811 11:58:44.029131 24647 tablet_service.cc:1514] Processing DeleteTablet for tablet 7cb231f466384a7fa4a57ed88980b87f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:58:44 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:50200
I20260811 11:58:44.029486 24753 tablet_replica.cc:330] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092: stopping tablet replica
I20260811 11:58:44.029614 24753 raft_consensus.cc:2229] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:58:44.029893 24753 raft_consensus.cc:2258] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:58:44.029994 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 24248
I20260811 11:58:44.029991 24755 tablet_replica.cc:330] T 7cb231f466384a7fa4a57ed88980b87f P ab7a1e62d2584140a350aa043e080e21: stopping tablet replica
I20260811 11:58:44.032209 24755 raft_consensus.cc:2229] T 7cb231f466384a7fa4a57ed88980b87f P ab7a1e62d2584140a350aa043e080e21 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:58:44.032531 24755 raft_consensus.cc:2258] T 7cb231f466384a7fa4a57ed88980b87f P ab7a1e62d2584140a350aa043e080e21 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:58:44.032891 24754 tablet_replica.cc:330] T 7cb231f466384a7fa4a57ed88980b87f P 6e657e4123504248aad5c023b128faa1: stopping tablet replica
I20260811 11:58:44.033241 24754 raft_consensus.cc:2229] T 7cb231f466384a7fa4a57ed88980b87f P 6e657e4123504248aad5c023b128faa1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:58:44.033540 24754 raft_consensus.cc:2258] T 7cb231f466384a7fa4a57ed88980b87f P 6e657e4123504248aad5c023b128faa1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:58:44.033921 24755 ts_tablet_manager.cc:1905] T 7cb231f466384a7fa4a57ed88980b87f P ab7a1e62d2584140a350aa043e080e21: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:58:44.035006 24754 ts_tablet_manager.cc:1905] T 7cb231f466384a7fa4a57ed88980b87f P 6e657e4123504248aad5c023b128faa1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:58:44.036199 24755 ts_tablet_manager.cc:1918] T 7cb231f466384a7fa4a57ed88980b87f P ab7a1e62d2584140a350aa043e080e21: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:58:44.036309 24755 log.cc:1196] T 7cb231f466384a7fa4a57ed88980b87f P ab7a1e62d2584140a350aa043e080e21: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/wal/wals/7cb231f466384a7fa4a57ed88980b87f
I20260811 11:58:44.036871 24755 ts_tablet_manager.cc:1939] T 7cb231f466384a7fa4a57ed88980b87f P ab7a1e62d2584140a350aa043e080e21: Deleting consensus metadata
I20260811 11:58:44.037957 24193 catalog_manager.cc:4841] TS ab7a1e62d2584140a350aa043e080e21 (127.6.143.131:33253): tablet 7cb231f466384a7fa4a57ed88980b87f (table test-table [id=82cc6e031b7447c2bea21ec38aa067f5]) successfully deleted
I20260811 11:58:44.038450 24754 ts_tablet_manager.cc:1918] T 7cb231f466384a7fa4a57ed88980b87f P 6e657e4123504248aad5c023b128faa1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:58:44.038554 24754 log.cc:1196] T 7cb231f466384a7fa4a57ed88980b87f P 6e657e4123504248aad5c023b128faa1: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/wal/wals/7cb231f466384a7fa4a57ed88980b87f
I20260811 11:58:44.038899 24754 ts_tablet_manager.cc:1939] T 7cb231f466384a7fa4a57ed88980b87f P 6e657e4123504248aad5c023b128faa1: Deleting consensus metadata
I20260811 11:58:44.040134 24190 catalog_manager.cc:4841] TS 6e657e4123504248aad5c023b128faa1 (127.6.143.130:41053): tablet 7cb231f466384a7fa4a57ed88980b87f (table test-table [id=82cc6e031b7447c2bea21ec38aa067f5]) successfully deleted
W20260811 11:58:44.042205 24191 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.143.129:41993 (error 108)
I20260811 11:58:44.042255 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 24423
I20260811 11:58:44.045094 24728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:58:44.044828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:41993 (local address 127.0.0.1:51064)
0811 11:58:44.044962 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 11:58:44.045029 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.129:41993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 11:58:44.045238 24191 catalog_manager.cc:4584] TS dc69aa9e3ea24eb6b8c4d3f92ad57092 (127.6.143.129:41993): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7cb231f466384a7fa4a57ed88980b87f: Network error: Client connection negotiation failed: client connection to 127.6.143.129:41993: connect: Connection refused (error 111)
I20260811 11:58:44.050179 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 24575
I20260811 11:58:44.056896 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 24173
I20260811 11:59:04.065383 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:46307
--webserver_interface=127.6.143.190
--webserver_port=33289
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:33967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:46307
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:04.179867 24757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:04.180150 24757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:04.180224 24757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:04.184159 24757 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 11:59:04.184202 24757 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 11:59:04.184218 24757 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 11:59:04.184267 24757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:59:04.184306 24757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:04.184343 24757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:04.184381 24757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:59:04.184445 24757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:59:04.189625 24757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:33967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:46307
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:46307
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=33289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:04.190666 24757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:04.191720 24757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:04.200206 24762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:04.200219 24767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:04.200431 24757 server_base.cc:1034] running on GCE node
W20260811 11:59:04.200265 24765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:04.201115 24757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:04.202195 24757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:04.203356 24757 hybrid_clock.cc:648] HybridClock initialized: now 1786449544203338 us; error 47 us; skew 500 ppm
Aug 11 11:59:04 dist-test-slave-zkpd krb5kdc[23570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449544, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:04.211481 24757 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:59:04.212558 24757 webserver.cc:460] Webserver started at http://127.6.143.190:33289/ using document root <none> and password file <none>
I20260811 11:59:04.213173 24757 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:04.213275 24757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:04.217338 24757 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 11:59:04.219645 24775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:04.220969 24757 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:59:04.221120 24757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "696c9ee4fe784fc39f4ab3e090399c7d"
format_stamp: "Formatted at 2026-08-11 11:58:22 on dist-test-slave-zkpd"
server_key: "d89de86cfcdf695666c0f4819218f007"
server_key_iv: "f50d9956f7d1755a31e50b0845fbc856"
server_key_version: "encryptionkey@0"
I20260811 11:59:04.221706 24757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:04.232607 24757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:04.235884 24757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:04.236143 24757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:04.243381 24757 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:46307
I20260811 11:59:04.243682 24832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:46307 every 8 connection(s)
I20260811 11:59:04.244537 24757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:59:04.248867 24833 sys_catalog.cc:263] Verifying existing consensus state
I20260811 11:59:04.250455 24833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d: Bootstrap starting.
I20260811 11:59:04.253387 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 24757
I20260811 11:59:04.253919 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:41993
--local_ip_for_outbound_sockets=127.6.143.129
--tserver_master_addrs=127.6.143.190:46307
--webserver_port=40139
--webserver_interface=127.6.143.129
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:33967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:59:04.269716 24833 log.cc:824] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:04.281800 24833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 11:59:04.282207 24833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d: Bootstrap complete.
I20260811 11:59:04.286468 24833 raft_consensus.cc:348] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696c9ee4fe784fc39f4ab3e090399c7d" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46307 } }
I20260811 11:59:04.286837 24833 raft_consensus.cc:729] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 696c9ee4fe784fc39f4ab3e090399c7d, State: Initialized, Role: FOLLOWER
I20260811 11:59:04.287340 24833 consensus_queue.cc:260] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696c9ee4fe784fc39f4ab3e090399c7d" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46307 } }
I20260811 11:59:04.287457 24833 raft_consensus.cc:388] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:59:04.287544 24833 raft_consensus.cc:482] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:59:04.287664 24833 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [term 2 FOLLOWER]: Advancing to term 3
I20260811 11:59:04.289746 24833 raft_consensus.cc:504] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696c9ee4fe784fc39f4ab3e090399c7d" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46307 } }
I20260811 11:59:04.290086 24833 leader_election.cc:302] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 696c9ee4fe784fc39f4ab3e090399c7d; no voters:
I20260811 11:59:04.290423 24833 leader_election.cc:288] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 11:59:04.290508 24837 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [term 3 FOLLOWER]: Leader election won for term 3
I20260811 11:59:04.290738 24837 raft_consensus.cc:686] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [term 3 LEADER]: Becoming Leader. State: Replica: 696c9ee4fe784fc39f4ab3e090399c7d, State: Running, Role: LEADER
I20260811 11:59:04.291095 24837 consensus_queue.cc:237] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696c9ee4fe784fc39f4ab3e090399c7d" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46307 } }
I20260811 11:59:04.291668 24833 sys_catalog.cc:564] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:59:04.292068 24837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 696c9ee4fe784fc39f4ab3e090399c7d. Latest consensus state: current_term: 3 leader_uuid: "696c9ee4fe784fc39f4ab3e090399c7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696c9ee4fe784fc39f4ab3e090399c7d" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46307 } } }
I20260811 11:59:04.292263 24837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:04.292469 24838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "696c9ee4fe784fc39f4ab3e090399c7d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "696c9ee4fe784fc39f4ab3e090399c7d" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 46307 } } }
I20260811 11:59:04.292645 24838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:04.292855 24841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:59:04.298203 24841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:59:04.299019 24841 catalog_manager.cc:1259] Loaded cluster ID: 0a31da5bc05f481ea0d7fe791ef87299
I20260811 11:59:04.299117 24841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:59:04.301093 24841 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:59:04.301792 24841 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d: Loaded TSK: 0
I20260811 11:59:04.301887 24841 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 696c9ee4fe784fc39f4ab3e090399c7d: Loaded TSK: 1
F20260811 11:59:04.302491 24841 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe12290f9ad google::LogMessage::Fail() at ??:0
@ 0x7fe12291288a google::LogMessage::SendToLog() at ??:0
@ 0x7fe12290f4de google::LogMessage::Flush() at ??:0
@ 0x7fe122910a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe1255d318d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fe1255cbf0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fe12d2747bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fe12d24e99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fe12d24f178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7fe12d290df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fe12d2af82b std::function<>::operator()() at ??:0
@ 0x7fe12d24eb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fe12d24ff3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe12d24c579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fe12d290640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fe124b613d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe124b61c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fe124b633f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fe124b52512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe1261966db start_thread at ??:0
@ 0x7fe12140d61f clone at ??:0
W20260811 11:59:04.390740 24835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:04.390982 24835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:04.391024 24835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:04.395282 24835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:04.395323 24835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:04.395377 24835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:59:04.400784 24835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:33967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:41993
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=40139
--tserver_master_addrs=127.6.143.190:46307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:04.401911 24835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:04.403013 24835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:04.412932 24886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:04.412990 24888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:04.413105 24885 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:04.413305 24835 server_base.cc:1034] running on GCE node
I20260811 11:59:04.413918 24835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:04.414769 24835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:04.416052 24835 hybrid_clock.cc:648] HybridClock initialized: now 1786449544416029 us; error 36 us; skew 500 ppm
Aug 11 11:59:04 dist-test-slave-zkpd krb5kdc[23570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449544, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:04.419649 24835 init.cc:377] Logged in from keytab as kudu/127.6.143.129@KRBTEST.COM (short username kudu)
I20260811 11:59:04.420850 24835 webserver.cc:460] Webserver started at http://127.6.143.129:40139/ using document root <none> and password file <none>
I20260811 11:59:04.421602 24835 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:04.421716 24835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:04.427399 24835 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 11:59:04.430409 24896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:04.432046 24835 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 11:59:04.432193 24835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092"
format_stamp: "Formatted at 2026-08-11 11:58:22 on dist-test-slave-zkpd"
server_key: "e9768ccb17b7d068dfe7aea47e95640c"
server_key_iv: "79cb1c18e444eb829807aed0ee51e2be"
server_key_version: "encryptionkey@0"
I20260811 11:59:04.432979 24835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:04.459811 24835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:04.463990 24835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:04.464215 24835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:04.464949 24835 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:59:04.466214 24903 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 11:59:04.470467 24835 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 11:59:04.470541 24835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260811 11:59:04.470605 24835 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 11:59:04.472963 24835 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 11:59:04.473023 24835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260811 11:59:04.473351 24903 tablet_bootstrap.cc:492] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092: Bootstrap starting.
I20260811 11:59:04.484068 24835 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:41993
I20260811 11:59:04.484213 25017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:41993 every 8 connection(s)
I20260811 11:59:04.485534 24835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:59:04.492369 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 24835
I20260811 11:59:04.492893 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:41053
--local_ip_for_outbound_sockets=127.6.143.130
--tserver_master_addrs=127.6.143.190:46307
--webserver_port=45449
--webserver_interface=127.6.143.130
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:33967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:59:04.494020 24903 log.cc:824] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:04.503050 24903 tablet_bootstrap.cc:492] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 11:59:04.503472 24903 tablet_bootstrap.cc:492] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092: Bootstrap complete.
I20260811 11:59:04.504967 24903 ts_tablet_manager.cc:1397] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092: Time spent bootstrapping tablet: real 0.032s user 0.019s sys 0.011s
I20260811 11:59:04.508152 24903 raft_consensus.cc:348] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 } } peers { permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 } } peers { permanent_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41993 } }
I20260811 11:59:04.508682 24903 raft_consensus.cc:729] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc69aa9e3ea24eb6b8c4d3f92ad57092, State: Initialized, Role: FOLLOWER
I20260811 11:59:04.509193 24903 consensus_queue.cc:260] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 } } peers { permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 } } peers { permanent_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41993 } }
I20260811 11:59:04.510310 24903 ts_tablet_manager.cc:1428] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
W20260811 11:59:04.568961 25020 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:59:04.487817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.129:40593)
0811 11:59:04.488484 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:04.488542 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:04.489578 (+ 1036us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:04.567381 (+ 77803us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":178,"threads_started":1}
W20260811 11:59:04.569562 25018 heartbeater.cc:644] Failed to heartbeat to 127.6.143.190:46307 (0 consecutive failures): Network error: Failed to ping master at 127.6.143.190:46307: Client connection negotiation failed: client connection to 127.6.143.190:46307: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 11:59:04.570372 25020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:04.570080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.129:35797)
0811 11:59:04.570224 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:04.570307 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 11:59:04.571060 25020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:04.570836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.129:58589)
0811 11:59:04.570952 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:04.571006 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 11:59:04.571247 25018 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 11:59:04.610224 25022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:04.610452 25022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:04.610522 25022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:04.614630 25022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:04.614686 25022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:04.614791 25022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:59:04.619797 25022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:33967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:41053
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=45449
--tserver_master_addrs=127.6.143.190:46307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:04.620841 25022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:04.621856 25022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:04.631242 25029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:04.631376 25030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:04.631399 25022 server_base.cc:1034] running on GCE node
W20260811 11:59:04.631531 25032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:04.632020 25022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:04.632592 25022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:04.633767 25022 hybrid_clock.cc:648] HybridClock initialized: now 1786449544633746 us; error 31 us; skew 500 ppm
Aug 11 11:59:04 dist-test-slave-zkpd krb5kdc[23570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449544, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:04.636525 25022 init.cc:377] Logged in from keytab as kudu/127.6.143.130@KRBTEST.COM (short username kudu)
I20260811 11:59:04.637380 25022 webserver.cc:460] Webserver started at http://127.6.143.130:45449/ using document root <none> and password file <none>
I20260811 11:59:04.637969 25022 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:04.638056 25022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:04.642030 25022 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 11:59:04.644058 25039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:04.645287 25022 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:59:04.645426 25022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "6e657e4123504248aad5c023b128faa1"
format_stamp: "Formatted at 2026-08-11 11:58:22 on dist-test-slave-zkpd"
server_key: "e24845d32925990f4b1eb464e62c85c3"
server_key_iv: "41cff7a9aaf750f98b47846238987975"
server_key_version: "encryptionkey@0"
I20260811 11:59:04.645962 25022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:04.660494 25022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:04.663177 25022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:04.663383 25022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:04.664040 25022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:59:04.664866 25022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:59:04.664937 25022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:04.665010 25022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:59:04.665055 25022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:04.674220 25022 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:41053
I20260811 11:59:04.674263 25164 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:41053 every 8 connection(s)
I20260811 11:59:04.675195 25022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
I20260811 11:59:04.678628 25168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:04.677199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.130:42977)
0811 11:59:04.677845 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:04.677949 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":117,"threads_started":1}
W20260811 11:59:04.679052 25166 heartbeater.cc:644] Failed to heartbeat to 127.6.143.190:46307 (0 consecutive failures): Network error: Failed to ping master at 127.6.143.190:46307: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
I20260811 11:59:04.679667 25168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:04.679469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.130:57971)
0811 11:59:04.679571 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:04.679615 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 11:59:04.680249 25168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:04.680068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.130:44421)
0811 11:59:04.680166 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:04.680205 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 11:59:04.680456 25166 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 11:59:04.680519 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 25022
I20260811 11:59:04.680959 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:33253
--local_ip_for_outbound_sockets=127.6.143.131
--tserver_master_addrs=127.6.143.190:46307
--webserver_port=46379
--webserver_interface=127.6.143.131
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:33967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:04.793530 25169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:04.793797 25169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:04.793869 25169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:04.797871 25169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:04.797919 25169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:04.798023 25169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:59:04.803362 25169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:33967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:33253
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=46379
--tserver_master_addrs=127.6.143.190:46307
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:04.804416 25169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:04.805436 25169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:04.815073 25175 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:04.815260 25177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:04.815164 25169 server_base.cc:1034] running on GCE node
W20260811 11:59:04.815086 25174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:04.815848 25169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:04.816514 25169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:04.817689 25169 hybrid_clock.cc:648] HybridClock initialized: now 1786449544817636 us; error 47 us; skew 500 ppm
Aug 11 11:59:04 dist-test-slave-zkpd krb5kdc[23570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449544, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:04.820504 25169 init.cc:377] Logged in from keytab as kudu/127.6.143.131@KRBTEST.COM (short username kudu)
I20260811 11:59:04.821336 25169 webserver.cc:460] Webserver started at http://127.6.143.131:46379/ using document root <none> and password file <none>
I20260811 11:59:04.821904 25169 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:04.821966 25169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:04.825762 25169 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260811 11:59:04.827948 25185 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:04.829241 25169 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:59:04.829334 25169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "ab7a1e62d2584140a350aa043e080e21"
format_stamp: "Formatted at 2026-08-11 11:58:22 on dist-test-slave-zkpd"
server_key: "215383abe6819f4e96dab778758d3ecd"
server_key_iv: "c6a77d2642167cad1e94bd5f002da8ab"
server_key_version: "encryptionkey@0"
I20260811 11:59:04.829846 25169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:04.858822 25169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:04.861642 25169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:04.861796 25169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:04.862313 25169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:59:04.863107 25169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:59:04.863154 25169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:04.863193 25169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:59:04.863209 25169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:04.872478 25169 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:33253
I20260811 11:59:04.872521 25307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:33253 every 8 connection(s)
I20260811 11:59:04.873504 25169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:59:04.876981 25311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:04.875507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.131:46977)
0811 11:59:04.876202 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:04.876305 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":111,"threads_started":1}
W20260811 11:59:04.877496 25308 heartbeater.cc:644] Failed to heartbeat to 127.6.143.190:46307 (0 consecutive failures): Network error: Failed to ping master at 127.6.143.190:46307: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
I20260811 11:59:04.878049 25311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:04.877853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.131:57863)
0811 11:59:04.877956 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:04.878000 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 11:59:04.878472 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 25169
I20260811 11:59:04.878724 25311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:04.878491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.131:47101)
0811 11:59:04.878637 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:04.878677 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 11:59:04.878922 25308 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 11:59:05.572388 25313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:05.571957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.129:42593)
0811 11:59:05.572216 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:05.572273 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":75,"threads_started":1}
I20260811 11:59:05.681386 25314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:05.681009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.130:56255)
0811 11:59:05.681232 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:05.681285 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":65,"threads_started":1}
I20260811 11:59:05.830753 25315 raft_consensus.cc:482] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:59:05.830952 25315 raft_consensus.cc:504] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 } } peers { permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 } } peers { permanent_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41993 } }
I20260811 11:59:05.832152 25315 leader_election.cc:288] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e657e4123504248aad5c023b128faa1 (127.6.143.130:41053), ab7a1e62d2584140a350aa043e080e21 (127.6.143.131:33253)
Aug 11 11:59:05 dist-test-slave-zkpd krb5kdc[23570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449544, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for kudu/127.6.143.130@KRBTEST.COM
Aug 11 11:59:05 dist-test-slave-zkpd krb5kdc[23570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449544, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for kudu/127.6.143.131@KRBTEST.COM
I20260811 11:59:05.846019 25317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:05.832576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:38561 (local address 127.6.143.130:41053)
0811 11:59:05.832982 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:05.833004 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:05.833049 (+ 45us) server_negotiation.cc:407] Connection header received
0811 11:59:05.833971 (+ 922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:05.833980 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:05.834455 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:05.834789 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:05.835944 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:05.836921 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:05.838243 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:05.838727 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:05.838935 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:05.841419 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:05.841424 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:05.841437 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:05.841465 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:05.843235 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:05.843774 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:05.843780 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:05.843785 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:05.843957 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:05.844302 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:05.844307 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:05.844309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:05.844842 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:05.844993 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:05.845573 (+ 580us) server_negotiation.cc:299] Negotiation successful
0811 11:59:05.845835 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":76,"threads_started":1}
I20260811 11:59:05.846009 25316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:05.832410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:33253 (local address 127.6.143.129:46711)
0811 11:59:05.832775 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:05.832791 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:05.832891 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:05.835292 (+ 2401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:05.835294 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:05.835467 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:59:05.835722 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:05.835736 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:05.837087 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:05.837095 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:05.838115 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:05.838125 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:05.838992 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:05.838996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:05.839074 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:05.839819 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:59:05.839832 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:59:05.841569 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:59:05.843788 (+ 2219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:05.843792 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:05.843794 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:05.844065 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:05.844413 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:05.844420 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:05.844424 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:05.844513 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:05.845230 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:59:05.845233 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:59:05.845310 (+ 77us) client_negotiation.cc:769] Sending connection context
0811 11:59:05.845443 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 11:59:05.845783 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"mutex_wait_us":56,"spinlock_wait_cycles":896,"thread_start_us":79,"threads_started":1}
I20260811 11:59:05.846112 25313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:05.832295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:41053 (local address 127.6.143.129:38561)
0811 11:59:05.832395 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:05.832409 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:05.832520 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:05.834868 (+ 2348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:05.834874 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:05.835249 (+ 375us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:59:05.835739 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:05.835750 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:05.837085 (+ 1335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:05.837088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:05.838116 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:05.838127 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:05.838867 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:05.838871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:05.839036 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:05.839658 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:59:05.839674 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:59:05.841168 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:59:05.843370 (+ 2202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:05.843376 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:05.843393 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:05.843645 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:05.844057 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:05.844061 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:05.844063 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:05.844200 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:05.844968 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:59:05.844974 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:59:05.845220 (+ 246us) client_negotiation.cc:769] Sending connection context
0811 11:59:05.845442 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 11:59:05.845879 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":244}
I20260811 11:59:05.846076 25318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:05.832625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:46711 (local address 127.6.143.131:33253)
0811 11:59:05.832982 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:05.833002 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:05.833049 (+ 47us) server_negotiation.cc:407] Connection header received
0811 11:59:05.833970 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:05.833980 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:05.834455 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:05.834789 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:05.835924 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:05.836934 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:05.838280 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:05.838866 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:05.839042 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:05.841763 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:05.841771 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:05.841788 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:05.841822 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:05.843654 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:05.844179 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:05.844185 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:05.844190 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:05.844309 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:05.844615 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:05.844622 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:05.844623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:05.845128 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:05.845264 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:05.845590 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 11:59:05.845879 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":108,"threads_started":1}
W20260811 11:59:05.848747 24900 leader_election.cc:341] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ab7a1e62d2584140a350aa043e080e21 (127.6.143.131:33253): Not found: Tablet not found: 7cb231f466384a7fa4a57ed88980b87f
W20260811 11:59:05.848924 24897 leader_election.cc:341] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6e657e4123504248aad5c023b128faa1 (127.6.143.130:41053): Not found: Tablet not found: 7cb231f466384a7fa4a57ed88980b87f
I20260811 11:59:05.849033 24897 leader_election.cc:302] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc69aa9e3ea24eb6b8c4d3f92ad57092; no voters: 6e657e4123504248aad5c023b128faa1, ab7a1e62d2584140a350aa043e080e21
I20260811 11:59:05.849364 25315 raft_consensus.cc:2726] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 11:59:05.879766 25319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:05.879401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.131:51881)
0811 11:59:05.879639 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:05.879686 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":78,"threads_started":1}
I20260811 11:59:06.573536 25320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:06.573128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.129:44037)
0811 11:59:06.573377 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:06.573442 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":80,"threads_started":1}
I20260811 11:59:06.682511 25321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:06.682103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.130:42705)
0811 11:59:06.682361 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:06.682425 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":94,"threads_started":1}
I20260811 11:59:06.880806 25322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:06.880419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.131:50381)
0811 11:59:06.880649 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:06.880703 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":70,"threads_started":1}
I20260811 11:59:07.574940 25323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:07.574464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.129:59649)
0811 11:59:07.574762 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:07.574828 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":79,"threads_started":1}
I20260811 11:59:07.683740 25324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:07.683310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.130:35739)
0811 11:59:07.683576 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:07.683640 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":74,"threads_started":1}
I20260811 11:59:07.740434 25325 raft_consensus.cc:482] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:59:07.740592 25325 raft_consensus.cc:504] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 } } peers { permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 } } peers { permanent_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41993 } }
I20260811 11:59:07.741117 25325 leader_election.cc:288] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e657e4123504248aad5c023b128faa1 (127.6.143.130:41053), ab7a1e62d2584140a350aa043e080e21 (127.6.143.131:33253)
W20260811 11:59:07.741968 24900 leader_election.cc:341] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ab7a1e62d2584140a350aa043e080e21 (127.6.143.131:33253): Not found: Tablet not found: 7cb231f466384a7fa4a57ed88980b87f
W20260811 11:59:07.742105 24897 leader_election.cc:341] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6e657e4123504248aad5c023b128faa1 (127.6.143.130:41053): Not found: Tablet not found: 7cb231f466384a7fa4a57ed88980b87f
I20260811 11:59:07.742197 24897 leader_election.cc:302] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc69aa9e3ea24eb6b8c4d3f92ad57092; no voters: 6e657e4123504248aad5c023b128faa1, ab7a1e62d2584140a350aa043e080e21
I20260811 11:59:07.742388 25325 raft_consensus.cc:2726] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 11:59:07.882247 25326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:07.881806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.131:41759)
0811 11:59:07.882091 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:07.882154 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":78,"threads_started":1}
I20260811 11:59:08.576201 25328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:08.575724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.129:43265)
0811 11:59:08.576048 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:08.576105 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":76,"threads_started":1}
I20260811 11:59:08.684827 25329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:08.684466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.130:49697)
0811 11:59:08.684691 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:08.684745 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":70,"threads_started":1}
I20260811 11:59:08.883534 25330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:08.882989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.131:52437)
0811 11:59:08.883388 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:08.883444 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":89,"threads_started":1}
I20260811 11:59:09.577523 25331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:09.577089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.129:43283)
0811 11:59:09.577367 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:09.577427 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":72,"threads_started":1}
I20260811 11:59:09.686079 25332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:09.685555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.130:46145)
0811 11:59:09.685914 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:09.685988 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":73,"threads_started":1}
I20260811 11:59:09.884816 25333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:09.884380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.131:59667)
0811 11:59:09.884652 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:09.884715 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":78,"threads_started":1}
I20260811 11:59:10.578683 25334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:10.578277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.129:48941)
0811 11:59:10.578531 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:10.578586 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":80,"threads_started":1}
I20260811 11:59:10.686623 25335 raft_consensus.cc:482] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:59:10.686832 25335 raft_consensus.cc:504] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 } } peers { permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 } } peers { permanent_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41993 } }
I20260811 11:59:10.687240 25336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:10.686844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.130:56365)
0811 11:59:10.687075 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:10.687133 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":83,"threads_started":1}
I20260811 11:59:10.687294 25335 leader_election.cc:288] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e657e4123504248aad5c023b128faa1 (127.6.143.130:41053), ab7a1e62d2584140a350aa043e080e21 (127.6.143.131:33253)
W20260811 11:59:10.688074 24897 leader_election.cc:341] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6e657e4123504248aad5c023b128faa1 (127.6.143.130:41053): Not found: Tablet not found: 7cb231f466384a7fa4a57ed88980b87f
W20260811 11:59:10.688184 24900 leader_election.cc:341] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ab7a1e62d2584140a350aa043e080e21 (127.6.143.131:33253): Not found: Tablet not found: 7cb231f466384a7fa4a57ed88980b87f
I20260811 11:59:10.688248 24897 leader_election.cc:302] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc69aa9e3ea24eb6b8c4d3f92ad57092; no voters: 6e657e4123504248aad5c023b128faa1, ab7a1e62d2584140a350aa043e080e21
I20260811 11:59:10.688400 25335 raft_consensus.cc:2726] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 11:59:10.885993 25337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:10.885580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.131:42519)
0811 11:59:10.885847 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:10.885903 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":89,"threads_started":1}
I20260811 11:59:11.580082 25338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:11.579548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.129:38363)
0811 11:59:11.579855 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:11.579966 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":106,"threads_started":1}
I20260811 11:59:11.688549 25339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:11.688116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.130:34763)
0811 11:59:11.688414 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:11.688465 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":85,"threads_started":1}
I20260811 11:59:11.887279 25340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:11.886819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.131:39181)
0811 11:59:11.887129 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:11.887180 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":91,"threads_started":1}
I20260811 11:59:12.581218 25341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:12.580812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.129:39093)
0811 11:59:12.581066 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:12.581118 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":86,"threads_started":1}
I20260811 11:59:12.689709 25342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:12.689299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.130:40099)
0811 11:59:12.689563 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:12.689617 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":95,"threads_started":1}
I20260811 11:59:12.888650 25343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:12.888182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.131:46159)
0811 11:59:12.888490 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:12.888552 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":83,"threads_started":1}
I20260811 11:59:13.563622 25344 raft_consensus.cc:482] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:59:13.563818 25344 raft_consensus.cc:504] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 } } peers { permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 } } peers { permanent_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41993 } }
I20260811 11:59:13.564383 25344 leader_election.cc:288] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e657e4123504248aad5c023b128faa1 (127.6.143.130:41053), ab7a1e62d2584140a350aa043e080e21 (127.6.143.131:33253)
W20260811 11:59:13.565166 24897 leader_election.cc:341] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6e657e4123504248aad5c023b128faa1 (127.6.143.130:41053): Not found: Tablet not found: 7cb231f466384a7fa4a57ed88980b87f
W20260811 11:59:13.565271 24900 leader_election.cc:341] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ab7a1e62d2584140a350aa043e080e21 (127.6.143.131:33253): Not found: Tablet not found: 7cb231f466384a7fa4a57ed88980b87f
I20260811 11:59:13.565338 24900 leader_election.cc:302] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc69aa9e3ea24eb6b8c4d3f92ad57092; no voters: 6e657e4123504248aad5c023b128faa1, ab7a1e62d2584140a350aa043e080e21
I20260811 11:59:13.565548 25344 raft_consensus.cc:2726] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 11:59:13.582297 25345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:13.581930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.129:45501)
0811 11:59:13.582160 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:13.582210 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":66,"threads_started":1}
I20260811 11:59:13.690835 25346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:13.690452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.130:42749)
0811 11:59:13.690694 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:13.690750 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":71,"threads_started":1}
I20260811 11:59:13.890043 25347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:13.889557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.131:39375)
0811 11:59:13.889833 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:13.889921 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":121,"threads_started":1}
I20260811 11:59:14.583694 25348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:14.583228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.129:35095)
0811 11:59:14.583533 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:14.583596 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":76,"threads_started":1}
I20260811 11:59:14.692142 25349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:14.691650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.130:43889)
0811 11:59:14.691997 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:14.692053 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":110,"threads_started":1}
I20260811 11:59:14.891307 25351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:14.890858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.131:38179)
0811 11:59:14.891155 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:14.891211 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":106,"threads_started":1}
I20260811 11:59:15.584930 25352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:15.584486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.129:58621)
0811 11:59:15.584779 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:15.584834 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":74,"threads_started":1}
I20260811 11:59:15.693259 25353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:15.692871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.130:47971)
0811 11:59:15.693121 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:15.693174 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":73,"threads_started":1}
I20260811 11:59:15.892550 25354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:15.892149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.131:53447)
0811 11:59:15.892399 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:15.892455 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":76,"threads_started":1}
I20260811 11:59:16.586146 25355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:16.585711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.129:40993)
0811 11:59:16.585991 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:16.586049 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":101,"threads_started":1}
I20260811 11:59:16.694422 25356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:16.693990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.130:44215)
0811 11:59:16.694268 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:16.694327 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":74,"threads_started":1}
I20260811 11:59:16.893882 25357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:16.893388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.131:45601)
0811 11:59:16.893717 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:16.893781 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":81,"threads_started":1}
I20260811 11:59:17.587425 25358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:17.586994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.129:59937)
0811 11:59:17.587269 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:17.587329 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":82,"threads_started":1}
I20260811 11:59:17.695597 25359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:17.695187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.130:40239)
0811 11:59:17.695450 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:17.695504 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":86,"threads_started":1}
I20260811 11:59:17.895155 25360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:17.894731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.131:50745)
0811 11:59:17.895001 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:17.895059 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":85,"threads_started":1}
I20260811 11:59:18.244043 25361 raft_consensus.cc:482] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:59:18.244177 25361 raft_consensus.cc:504] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e657e4123504248aad5c023b128faa1" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41053 } } peers { permanent_uuid: "ab7a1e62d2584140a350aa043e080e21" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 33253 } } peers { permanent_uuid: "dc69aa9e3ea24eb6b8c4d3f92ad57092" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 41993 } }
I20260811 11:59:18.244556 25361 leader_election.cc:288] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 6e657e4123504248aad5c023b128faa1 (127.6.143.130:41053), ab7a1e62d2584140a350aa043e080e21 (127.6.143.131:33253)
W20260811 11:59:18.245396 24897 leader_election.cc:341] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6e657e4123504248aad5c023b128faa1 (127.6.143.130:41053): Not found: Tablet not found: 7cb231f466384a7fa4a57ed88980b87f
W20260811 11:59:18.245576 24900 leader_election.cc:341] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer ab7a1e62d2584140a350aa043e080e21 (127.6.143.131:33253): Not found: Tablet not found: 7cb231f466384a7fa4a57ed88980b87f
I20260811 11:59:18.245668 24900 leader_election.cc:302] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dc69aa9e3ea24eb6b8c4d3f92ad57092; no voters: 6e657e4123504248aad5c023b128faa1, ab7a1e62d2584140a350aa043e080e21
I20260811 11:59:18.245919 25361 raft_consensus.cc:2726] T 7cb231f466384a7fa4a57ed88980b87f P dc69aa9e3ea24eb6b8c4d3f92ad57092 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 11:59:18.588837 25363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:18.588274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.129:38771)
0811 11:59:18.588667 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:18.588736 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":97,"threads_started":1}
I20260811 11:59:18.696863 25364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:18.696436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.130:41133)
0811 11:59:18.696685 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:18.696740 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":79,"threads_started":1}
I20260811 11:59:18.896451 25365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:18.895998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.131:53527)
0811 11:59:18.896290 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:18.896348 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":100,"threads_started":1}
I20260811 11:59:19.590296 25366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:19.589849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.129:33909)
0811 11:59:19.590132 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:19.590195 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":122,"threads_started":1}
I20260811 11:59:19.698194 25367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:19.697773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.130:53003)
0811 11:59:19.698028 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:19.698081 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":94,"threads_started":1}
I20260811 11:59:19.885269 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 24835
I20260811 11:59:19.895311 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 25022
I20260811 11:59:19.898554 25368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:19.897579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:46307 (local address 127.6.143.131:55671)
0811 11:59:19.898341 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:19.898418 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.190:46307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":70,"threads_started":1}
I20260811 11:59:19.903522 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 25169
2026-08-11T11:59:19Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59315 ms)
[----------] 30 tests from SecurityITest (240519 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:59:19 dist-test-slave-zkpd krb5kdc[25371](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:59:19 dist-test-slave-zkpd krb5kdc[25371](info): set up 2 sockets
Aug 11 11:59:19 dist-test-slave-zkpd krb5kdc[25371](info): commencing operation
krb5kdc: starting...
W20260811 11:59:21.027659 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:59:21 dist-test-slave-zkpd krb5kdc[25371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:59:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:59:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
I20260811 11:59:21.155761 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:45309
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:35583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:45309
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:21.270552 25385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:21.270857 25385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:21.270960 25385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:21.275103 25385 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:59:21.275195 25385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:21.275233 25385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:21.275271 25385 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:59:21.275301 25385 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:59:21.280452 25385 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:35583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:45309
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:45309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:21.281478 25385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:21.282487 25385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 11:59:21.291281 25385 server_base.cc:1034] running on GCE node
W20260811 11:59:21.291263 25391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:21.291363 25390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:21.291250 25393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:21.292130 25385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:21.293200 25385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:21.294399 25385 hybrid_clock.cc:648] HybridClock initialized: now 1786449561294366 us; error 43 us; skew 500 ppm
Aug 11 11:59:21 dist-test-slave-zkpd krb5kdc[25371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449561, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:21.297381 25385 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:59:21.298326 25385 webserver.cc:460] Webserver started at http://127.6.143.190:34617/ using document root <none> and password file <none>
I20260811 11:59:21.298936 25385 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:21.299036 25385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:21.299286 25385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:21.301466 25385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "6dc48d0de64544c28eecf257c552f882"
format_stamp: "Formatted at 2026-08-11 11:59:21 on dist-test-slave-zkpd"
server_key: "1c5a2749dbc500f5bd8cd5c0b1cf35d3"
server_key_iv: "ccbb3298a0b3237a91a75b1bb79d5a7c"
server_key_version: "encryptionkey@0"
I20260811 11:59:21.302063 25385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "6dc48d0de64544c28eecf257c552f882"
format_stamp: "Formatted at 2026-08-11 11:59:21 on dist-test-slave-zkpd"
server_key: "1c5a2749dbc500f5bd8cd5c0b1cf35d3"
server_key_iv: "ccbb3298a0b3237a91a75b1bb79d5a7c"
server_key_version: "encryptionkey@0"
I20260811 11:59:21.306057 25385 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 11:59:21.308923 25400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:21.310179 25385 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:59:21.310348 25385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "6dc48d0de64544c28eecf257c552f882"
format_stamp: "Formatted at 2026-08-11 11:59:21 on dist-test-slave-zkpd"
server_key: "1c5a2749dbc500f5bd8cd5c0b1cf35d3"
server_key_iv: "ccbb3298a0b3237a91a75b1bb79d5a7c"
server_key_version: "encryptionkey@0"
I20260811 11:59:21.310456 25385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:21.336109 25385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:21.339453 25385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:21.339665 25385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:21.347092 25385 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:45309
I20260811 11:59:21.347163 25456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:45309 every 8 connection(s)
I20260811 11:59:21.348133 25385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:59:21.350939 25457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:21.352221 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 25385
I20260811 11:59:21.352351 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:59:21.352697 6718 external_mini_cluster.cc:1428] Setting key 36700d63f1ef2adf97a6ffea9be51ff9
I20260811 11:59:21.357686 25457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dc48d0de64544c28eecf257c552f882: Bootstrap starting.
I20260811 11:59:21.360556 25457 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6dc48d0de64544c28eecf257c552f882: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:21.361522 25457 log.cc:824] T 00000000000000000000000000000000 P 6dc48d0de64544c28eecf257c552f882: Log is configured to *not* fsync() on all Append() calls
Aug 11 11:59:21 dist-test-slave-zkpd krb5kdc[25371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:59:21.363669 25457 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6dc48d0de64544c28eecf257c552f882: No bootstrap required, opened a new log
I20260811 11:59:21.367281 25460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:21.354122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55004 (local address 127.6.143.190:45309)
0811 11:59:21.354560 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:21.354576 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:21.354612 (+ 36us) server_negotiation.cc:407] Connection header received
0811 11:59:21.355263 (+ 651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:21.355280 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:21.355598 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:21.355882 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:21.356816 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:21.357782 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:21.359020 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:21.359494 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:21.359623 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:21.362564 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:21.362569 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:21.362583 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:21.362610 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:21.364469 (+ 1859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:21.364965 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:21.364970 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:21.364975 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:21.365083 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:21.365381 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:21.365385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:21.365387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:21.365773 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:21.365975 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:21.366288 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 11:59:21.366566 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":148,"threads_started":1}
I20260811 11:59:21.367292 25457 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6dc48d0de64544c28eecf257c552f882 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dc48d0de64544c28eecf257c552f882" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 45309 } }
I20260811 11:59:21.367643 25457 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6dc48d0de64544c28eecf257c552f882 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:21.367728 25457 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6dc48d0de64544c28eecf257c552f882 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6dc48d0de64544c28eecf257c552f882, State: Initialized, Role: FOLLOWER
I20260811 11:59:21.368219 25457 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6dc48d0de64544c28eecf257c552f882 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dc48d0de64544c28eecf257c552f882" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 45309 } }
I20260811 11:59:21.368372 25457 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6dc48d0de64544c28eecf257c552f882 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:59:21.368471 25457 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6dc48d0de64544c28eecf257c552f882 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:59:21.368575 25457 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6dc48d0de64544c28eecf257c552f882 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:21.369454 25457 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6dc48d0de64544c28eecf257c552f882 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dc48d0de64544c28eecf257c552f882" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 45309 } }
I20260811 11:59:21.369871 25457 leader_election.cc:302] T 00000000000000000000000000000000 P 6dc48d0de64544c28eecf257c552f882 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6dc48d0de64544c28eecf257c552f882; no voters:
I20260811 11:59:21.370177 25457 leader_election.cc:288] T 00000000000000000000000000000000 P 6dc48d0de64544c28eecf257c552f882 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:59:21.370282 25462 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6dc48d0de64544c28eecf257c552f882 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:21.370540 25462 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6dc48d0de64544c28eecf257c552f882 [term 1 LEADER]: Becoming Leader. State: Replica: 6dc48d0de64544c28eecf257c552f882, State: Running, Role: LEADER
I20260811 11:59:21.370905 25462 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6dc48d0de64544c28eecf257c552f882 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dc48d0de64544c28eecf257c552f882" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 45309 } }
I20260811 11:59:21.371378 25457 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6dc48d0de64544c28eecf257c552f882 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:59:21.372714 25464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dc48d0de64544c28eecf257c552f882 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6dc48d0de64544c28eecf257c552f882. Latest consensus state: current_term: 1 leader_uuid: "6dc48d0de64544c28eecf257c552f882" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dc48d0de64544c28eecf257c552f882" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 45309 } } }
I20260811 11:59:21.372769 25463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6dc48d0de64544c28eecf257c552f882 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6dc48d0de64544c28eecf257c552f882" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6dc48d0de64544c28eecf257c552f882" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 45309 } } }
I20260811 11:59:21.372856 25463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dc48d0de64544c28eecf257c552f882 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:21.372856 25464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6dc48d0de64544c28eecf257c552f882 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:21.373384 25472 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:59:21.376641 25472 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:59:21.382159 25472 catalog_manager.cc:1347] Generated new cluster ID: b1997db1cdd44b8c85e12b6b66762125
I20260811 11:59:21.382222 25472 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:59:21.394572 25472 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:59:21.395709 25472 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:59:21.408733 25472 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6dc48d0de64544c28eecf257c552f882: Generated new TSK 0
I20260811 11:59:21.409543 25472 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
I20260811 11:59:21.446388 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:0
--local_ip_for_outbound_sockets=127.6.143.129
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:45309
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:35583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:21.563512 25483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:21.563764 25483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:21.563805 25483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:21.567830 25483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:21.567883 25483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:21.568034 25483 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:59:21.573267 25483 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:35583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:45309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25483
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:21.574301 25483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:21.575292 25483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:21.584630 25491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:21.584664 25489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:21.584648 25488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:21.584877 25483 server_base.cc:1034] running on GCE node
I20260811 11:59:21.585430 25483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:21.586056 25483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:21.587244 25483 hybrid_clock.cc:648] HybridClock initialized: now 1786449561587205 us; error 42 us; skew 500 ppm
Aug 11 11:59:21 dist-test-slave-zkpd krb5kdc[25371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449561, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:21.590307 25483 init.cc:377] Logged in from keytab as kudu/127.6.143.129@KRBTEST.COM (short username kudu)
I20260811 11:59:21.591177 25483 webserver.cc:460] Webserver started at http://127.6.143.129:43305/ using document root <none> and password file <none>
I20260811 11:59:21.591799 25483 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:21.591902 25483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:21.592198 25483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:21.595166 25483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-0/data/instance:
uuid: "9d43cb0db93d43cda7a53e720c870cf2"
format_stamp: "Formatted at 2026-08-11 11:59:21 on dist-test-slave-zkpd"
server_key: "8dd1e092b6fe629f5846e09dd1edc053"
server_key_iv: "7b7be800f95e7776a06e03024031cc4d"
server_key_version: "encryptionkey@0"
I20260811 11:59:21.596055 25483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance:
uuid: "9d43cb0db93d43cda7a53e720c870cf2"
format_stamp: "Formatted at 2026-08-11 11:59:21 on dist-test-slave-zkpd"
server_key: "8dd1e092b6fe629f5846e09dd1edc053"
server_key_iv: "7b7be800f95e7776a06e03024031cc4d"
server_key_version: "encryptionkey@0"
I20260811 11:59:21.600188 25483 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.002s
I20260811 11:59:21.602715 25499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:21.604058 25483 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:59:21.604192 25483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "9d43cb0db93d43cda7a53e720c870cf2"
format_stamp: "Formatted at 2026-08-11 11:59:21 on dist-test-slave-zkpd"
server_key: "8dd1e092b6fe629f5846e09dd1edc053"
server_key_iv: "7b7be800f95e7776a06e03024031cc4d"
server_key_version: "encryptionkey@0"
I20260811 11:59:21.604337 25483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:21.629339 25483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:21.632527 25483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:21.632773 25483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:21.633533 25483 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:59:21.634516 25483 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:59:21.634600 25483 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:21.634683 25483 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:59:21.634732 25483 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:21.644665 25483 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:35627
I20260811 11:59:21.644692 25627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:35627 every 8 connection(s)
I20260811 11:59:21.645625 25483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:59:21.653589 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 25483
I20260811 11:59:21.653786 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance
I20260811 11:59:21.654121 6718 external_mini_cluster.cc:1428] Setting key a7fbcab89cd448b5726ccab7fbc7ea79
Aug 11 11:59:21 dist-test-slave-zkpd krb5kdc[25371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449561, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:59:21.660701 25460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:21.647566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:51681 (local address 127.6.143.190:45309)
0811 11:59:21.647712 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:21.647716 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:21.648686 (+ 970us) server_negotiation.cc:407] Connection header received
0811 11:59:21.650006 (+ 1320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:21.650009 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:21.650066 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:21.650180 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:21.651375 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:21.652001 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:21.653183 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:21.653429 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:21.653544 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:21.656515 (+ 2971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:21.656522 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:21.656528 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:21.656563 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:21.658194 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:21.658833 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:21.658836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:21.658837 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:21.658912 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:21.659343 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:21.659345 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:21.659346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:21.659595 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:21.659700 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:21.660285 (+ 585us) server_negotiation.cc:299] Negotiation successful
0811 11:59:21.660458 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 11:59:21.661422 25630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:21.648056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:45309 (local address 127.6.143.129:51681)
0811 11:59:21.648527 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:21.648570 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:21.649775 (+ 1205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:21.650364 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:21.650372 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:21.650735 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:59:21.651219 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:21.651233 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:21.652141 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:21.652145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:21.653048 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:21.653059 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:21.653601 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:21.653605 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:21.653841 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:21.654570 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:59:21.654586 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:59:21.656323 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:59:21.658320 (+ 1997us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:21.658329 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:21.658347 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:21.658727 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:21.659010 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:21.659013 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:21.659014 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:21.659220 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:21.659708 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:59:21.659713 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:59:21.660029 (+ 316us) client_negotiation.cc:769] Sending connection context
0811 11:59:21.660202 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 11:59:21.660476 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":121,"threads_started":1}
I20260811 11:59:21.662767 25628 heartbeater.cc:347] Connected to a master server at 127.6.143.190:45309
I20260811 11:59:21.663009 25628 heartbeater.cc:464] Registering TS with master...
I20260811 11:59:21.663522 25628 heartbeater.cc:511] Master 127.6.143.190:45309 requested a full tablet report, sending...
I20260811 11:59:21.665197 25419 ts_manager.cc:194] Registered new tserver with Master: 9d43cb0db93d43cda7a53e720c870cf2 (127.6.143.129:35627)
WARNING: no policy specified for kudu/127.6.143.130@KRBTEST.COM; defaulting to no policy
I20260811 11:59:21.666419 25419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.129@KRBTEST.COM'} at 127.6.143.129:51681
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
I20260811 11:59:21.686813 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:0
--local_ip_for_outbound_sockets=127.6.143.130
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:45309
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:35583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:21.800001 25635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:21.800267 25635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:21.800310 25635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:21.804388 25635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:21.804458 25635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:21.804518 25635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:59:21.810160 25635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:35583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:45309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:21.811297 25635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:21.812419 25635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:21.822641 25642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:21.822755 25635 server_base.cc:1034] running on GCE node
W20260811 11:59:21.822641 25644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:21.822651 25641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:21.823400 25635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:21.824080 25635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:21.825320 25635 hybrid_clock.cc:648] HybridClock initialized: now 1786449561825273 us; error 50 us; skew 500 ppm
Aug 11 11:59:21 dist-test-slave-zkpd krb5kdc[25371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449561, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:21.828351 25635 init.cc:377] Logged in from keytab as kudu/127.6.143.130@KRBTEST.COM (short username kudu)
I20260811 11:59:21.829236 25635 webserver.cc:460] Webserver started at http://127.6.143.130:42539/ using document root <none> and password file <none>
I20260811 11:59:21.829809 25635 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:21.829903 25635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:21.830137 25635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:21.832307 25635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-1/data/instance:
uuid: "27673094e2894a5fae1b560d43dc531a"
format_stamp: "Formatted at 2026-08-11 11:59:21 on dist-test-slave-zkpd"
server_key: "038a6cca99454b422e8241b59ea6065b"
server_key_iv: "f7103c0d5c70c9a500a80a8705d92570"
server_key_version: "encryptionkey@0"
I20260811 11:59:21.832885 25635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance:
uuid: "27673094e2894a5fae1b560d43dc531a"
format_stamp: "Formatted at 2026-08-11 11:59:21 on dist-test-slave-zkpd"
server_key: "038a6cca99454b422e8241b59ea6065b"
server_key_iv: "f7103c0d5c70c9a500a80a8705d92570"
server_key_version: "encryptionkey@0"
I20260811 11:59:21.836674 25635 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 11:59:21.839056 25653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:21.840339 25635 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 11:59:21.840468 25635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "27673094e2894a5fae1b560d43dc531a"
format_stamp: "Formatted at 2026-08-11 11:59:21 on dist-test-slave-zkpd"
server_key: "038a6cca99454b422e8241b59ea6065b"
server_key_iv: "f7103c0d5c70c9a500a80a8705d92570"
server_key_version: "encryptionkey@0"
I20260811 11:59:21.840577 25635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:21.858333 25635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:21.861622 25635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:21.861853 25635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:21.862505 25635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:59:21.863412 25635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:59:21.863488 25635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:21.863559 25635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:59:21.863610 25635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:21.873282 25635 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:46291
I20260811 11:59:21.873369 25778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:46291 every 8 connection(s)
I20260811 11:59:21.874281 25635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
I20260811 11:59:21.884145 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 25635
I20260811 11:59:21.884300 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance
Aug 11 11:59:21 dist-test-slave-zkpd krb5kdc[25371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449561, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:59:21.884654 6718 external_mini_cluster.cc:1428] Setting key 29a046e0b36f616804a86b9fb48c2c71
I20260811 11:59:21.889258 25460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:21.876250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:38811 (local address 127.6.143.190:45309)
0811 11:59:21.876391 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:21.876395 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:21.877207 (+ 812us) server_negotiation.cc:407] Connection header received
0811 11:59:21.878668 (+ 1461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:21.878672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:21.878748 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:21.878836 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:21.880152 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:21.880741 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:21.882003 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:21.882257 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:21.882380 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:21.885116 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:21.885119 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:21.885121 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:21.885147 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:21.886698 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:21.887394 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:21.887396 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:21.887398 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:21.887472 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:21.887927 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:21.887931 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:21.887933 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:21.888133 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:21.888213 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:21.888890 (+ 677us) server_negotiation.cc:299] Negotiation successful
0811 11:59:21.889033 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 11:59:21.890038 25781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:21.876592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:45309 (local address 127.6.143.130:38811)
0811 11:59:21.877045 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:21.877089 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:21.878430 (+ 1341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:21.879013 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:21.879022 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:21.879414 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:59:21.880000 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:21.880017 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:21.880888 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:21.880892 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:21.881859 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:21.881870 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:21.882433 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:21.882437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:21.882614 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:21.883225 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:59:21.883243 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:59:21.884936 (+ 1693us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:59:21.886827 (+ 1891us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:21.886837 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:21.886857 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:21.887281 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:21.887568 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:21.887572 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:21.887574 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:21.887780 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:21.888255 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:59:21.888265 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:59:21.888618 (+ 353us) client_negotiation.cc:769] Sending connection context
0811 11:59:21.888791 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 11:59:21.889054 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":100,"threads_started":1}
I20260811 11:59:21.891333 25779 heartbeater.cc:347] Connected to a master server at 127.6.143.190:45309
I20260811 11:59:21.891610 25779 heartbeater.cc:464] Registering TS with master...
I20260811 11:59:21.892282 25779 heartbeater.cc:511] Master 127.6.143.190:45309 requested a full tablet report, sending...
I20260811 11:59:21.893540 25419 ts_manager.cc:194] Registered new tserver with Master: 27673094e2894a5fae1b560d43dc531a (127.6.143.130:46291)
I20260811 11:59:21.894160 25419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.130@KRBTEST.COM'} at 127.6.143.130:38811
WARNING: no policy specified for kudu/127.6.143.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
I20260811 11:59:21.914126 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:0
--local_ip_for_outbound_sockets=127.6.143.131
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:45309
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:35583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:22.025823 25784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:22.026113 25784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:22.026226 25784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:22.030386 25784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:22.030470 25784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:22.030556 25784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:59:22.035794 25784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:35583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:45309
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:22.036859 25784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:22.037886 25784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:22.047394 25792 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:22.047412 25791 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:22.047602 25784 server_base.cc:1034] running on GCE node
W20260811 11:59:22.047394 25794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:22.048153 25784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:22.048784 25784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:22.049993 25784 hybrid_clock.cc:648] HybridClock initialized: now 1786449562049980 us; error 29 us; skew 500 ppm
Aug 11 11:59:22 dist-test-slave-zkpd krb5kdc[25371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449562, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:22.052742 25784 init.cc:377] Logged in from keytab as kudu/127.6.143.131@KRBTEST.COM (short username kudu)
I20260811 11:59:22.053613 25784 webserver.cc:460] Webserver started at http://127.6.143.131:42985/ using document root <none> and password file <none>
I20260811 11:59:22.054153 25784 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:22.054211 25784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:22.054371 25784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:22.056516 25784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-2/data/instance:
uuid: "8da9337ee7fd4f6c8f0baea3444aabf6"
format_stamp: "Formatted at 2026-08-11 11:59:22 on dist-test-slave-zkpd"
server_key: "acf8d8d6409deb82ac33f12ea02e9ba7"
server_key_iv: "85c5390d230dd7ae60b7b1fa137e1aa3"
server_key_version: "encryptionkey@0"
I20260811 11:59:22.057016 25784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance:
uuid: "8da9337ee7fd4f6c8f0baea3444aabf6"
format_stamp: "Formatted at 2026-08-11 11:59:22 on dist-test-slave-zkpd"
server_key: "acf8d8d6409deb82ac33f12ea02e9ba7"
server_key_iv: "85c5390d230dd7ae60b7b1fa137e1aa3"
server_key_version: "encryptionkey@0"
I20260811 11:59:22.060566 25784 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 11:59:22.062880 25803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:22.064144 25784 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 11:59:22.064263 25784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "8da9337ee7fd4f6c8f0baea3444aabf6"
format_stamp: "Formatted at 2026-08-11 11:59:22 on dist-test-slave-zkpd"
server_key: "acf8d8d6409deb82ac33f12ea02e9ba7"
server_key_iv: "85c5390d230dd7ae60b7b1fa137e1aa3"
server_key_version: "encryptionkey@0"
I20260811 11:59:22.064359 25784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:22.079205 25784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:22.082036 25784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:22.082229 25784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:22.082784 25784 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:59:22.083586 25784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:59:22.083633 25784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:22.083702 25784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:59:22.083741 25784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:22.093312 25784 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:36319
I20260811 11:59:22.093331 25931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:36319 every 8 connection(s)
I20260811 11:59:22.094236 25784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:59:22.100678 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 25784
I20260811 11:59:22.100827 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance
I20260811 11:59:22.101121 6718 external_mini_cluster.cc:1428] Setting key 86d2f2fc6ab7c1a88619db048a04b18d
Aug 11 11:59:22 dist-test-slave-zkpd krb5kdc[25371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449562, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:59:22.108148 25460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:22.096204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:33203 (local address 127.6.143.190:45309)
0811 11:59:22.096360 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:22.096365 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:22.097231 (+ 866us) server_negotiation.cc:407] Connection header received
0811 11:59:22.098455 (+ 1224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:22.098458 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:22.098503 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:22.098578 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:22.099767 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.100350 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.101659 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.101897 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.102040 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:22.104868 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:22.104871 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:22.104874 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:22.104906 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:22.106150 (+ 1244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:22.106652 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:22.106655 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:22.106657 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:22.106735 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:22.107086 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:22.107090 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:22.107092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:22.107267 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:22.107376 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:22.107835 (+ 459us) server_negotiation.cc:299] Negotiation successful
0811 11:59:22.107986 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 11:59:22.108964 25936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:22.096604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:45309 (local address 127.6.143.131:33203)
0811 11:59:22.097083 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:22.097125 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:22.098262 (+ 1137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:22.098788 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:22.098795 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:22.099165 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:59:22.099638 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:22.099651 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.100480 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.100483 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:22.101522 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:22.101533 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.102148 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.102156 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:22.102411 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:22.103112 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:59:22.103138 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:59:22.104708 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:59:22.106270 (+ 1562us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:22.106276 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:22.106291 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:22.106539 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:22.106836 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:22.106839 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:22.106842 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:22.106987 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:22.107376 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:59:22.107383 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:59:22.107608 (+ 225us) client_negotiation.cc:769] Sending connection context
0811 11:59:22.107795 (+ 187us) client_negotiation.cc:240] Negotiation successful
0811 11:59:22.108128 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":118,"threads_started":1}
I20260811 11:59:22.110072 25932 heartbeater.cc:347] Connected to a master server at 127.6.143.190:45309
I20260811 11:59:22.110313 25932 heartbeater.cc:464] Registering TS with master...
I20260811 11:59:22.110795 25932 heartbeater.cc:511] Master 127.6.143.190:45309 requested a full tablet report, sending...
I20260811 11:59:22.111858 25419 ts_manager.cc:194] Registered new tserver with Master: 8da9337ee7fd4f6c8f0baea3444aabf6 (127.6.143.131:36319)
I20260811 11:59:22.112506 25419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.131@KRBTEST.COM'} at 127.6.143.131:33203
I20260811 11:59:22.115108 6718 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:59:22.123960 25460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:22.117094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55012 (local address 127.6.143.190:45309)
0811 11:59:22.117235 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:22.117240 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:22.117374 (+ 134us) server_negotiation.cc:407] Connection header received
0811 11:59:22.117476 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:22.117478 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:22.117516 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:22.117580 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:22.118213 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.118693 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.119482 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.119680 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.119762 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:22.121035 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:22.121038 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:22.121040 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:22.121061 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:22.122473 (+ 1412us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:22.122827 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:22.122829 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:22.122831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:22.122900 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:22.123129 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:22.123131 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:22.123131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:22.123337 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:22.123440 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:22.123641 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 11:59:22.123731 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 11:59:22.126632 25419 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55012:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:59:22.129837 25419 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:59:22.145426 25950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:22.139093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55396 (local address 127.6.143.129:35627)
0811 11:59:22.139673 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:22.139695 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:22.139720 (+ 25us) server_negotiation.cc:407] Connection header received
0811 11:59:22.139806 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:22.139812 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:22.140096 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:22.140293 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:22.140619 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.141866 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.143843 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.144711 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.144846 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:22.145007 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:22.145147 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 11:59:22.145260 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":88,"threads_started":1}
I20260811 11:59:22.145442 25949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:22.138665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:35627 (local address 127.0.0.1:55396)
0811 11:59:22.139479 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:22.139498 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:22.139604 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:22.140275 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:22.140281 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:22.140304 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:22.140473 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:22.140483 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.142033 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.142039 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:22.143686 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:22.143699 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.144846 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.144850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:22.144956 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:22.144992 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 11:59:22.145192 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 11:59:22.145293 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"mutex_wait_us":167,"thread_start_us":131,"threads_started":1}
I20260811 11:59:22.145781 25952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:22.139729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40182 (local address 127.6.143.131:36319)
0811 11:59:22.140196 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:22.140208 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:22.140230 (+ 22us) server_negotiation.cc:407] Connection header received
0811 11:59:22.140303 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:22.140326 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:22.140530 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:22.140688 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:22.141027 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.141821 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.143828 (+ 2007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.144747 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.144889 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:22.145035 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:22.145520 (+ 485us) server_negotiation.cc:299] Negotiation successful
0811 11:59:22.145627 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":162,"threads_started":1}
I20260811 11:59:22.145854 25951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:22.139260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:36319 (local address 127.0.0.1:40182)
0811 11:59:22.139982 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:22.139995 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:22.140079 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:22.140758 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:22.140762 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:22.140781 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:22.140888 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:22.140894 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.142026 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.142029 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:22.143686 (+ 1657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:22.143699 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.145100 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.145105 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:22.145202 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:22.145241 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 11:59:22.145315 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 11:59:22.145404 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"mutex_wait_us":145,"thread_start_us":99,"threads_started":1}
I20260811 11:59:22.146085 25947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:22.138533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:46291 (local address 127.0.0.1:38004)
0811 11:59:22.139161 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:22.139210 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:22.139349 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:22.140537 (+ 1188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:22.140542 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:22.140563 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:22.140666 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:22.140674 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.142054 (+ 1380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.142057 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:22.143538 (+ 1481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:22.143551 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.144916 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.144920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:22.145015 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:22.145040 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 11:59:22.145099 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 11:59:22.145226 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"mutex_wait_us":182,"thread_start_us":61,"threads_started":1}
I20260811 11:59:22.146708 25948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:22.138660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38004 (local address 127.6.143.130:46291)
0811 11:59:22.139953 (+ 1293us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:22.139966 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:22.139991 (+ 25us) server_negotiation.cc:407] Connection header received
0811 11:59:22.140080 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:22.140086 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:22.140326 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:22.140538 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:22.140811 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.141912 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.143688 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.144756 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.146304 (+ 1548us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:22.146437 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:22.146482 (+ 45us) server_negotiation.cc:299] Negotiation successful
0811 11:59:22.146554 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1112,"thread_start_us":133,"threads_started":1}
I20260811 11:59:22.148746 25859 tablet_service.cc:1467] Processing CreateTablet for tablet 40223dbcbbc54f3c88d0902c7012d0d9 (DEFAULT_TABLE table=test-table [id=9d74f0d2d258493fbad39d849b367577]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:59:22.148994 25556 tablet_service.cc:1467] Processing CreateTablet for tablet 40223dbcbbc54f3c88d0902c7012d0d9 (DEFAULT_TABLE table=test-table [id=9d74f0d2d258493fbad39d849b367577]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:59:22.149744 25859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40223dbcbbc54f3c88d0902c7012d0d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:22.149477 25704 tablet_service.cc:1467] Processing CreateTablet for tablet 40223dbcbbc54f3c88d0902c7012d0d9 (DEFAULT_TABLE table=test-table [id=9d74f0d2d258493fbad39d849b367577]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:59:22.150131 25556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40223dbcbbc54f3c88d0902c7012d0d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:22.150493 25704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 40223dbcbbc54f3c88d0902c7012d0d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:22.155573 25953 tablet_bootstrap.cc:492] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6: Bootstrap starting.
I20260811 11:59:22.157706 25953 tablet_bootstrap.cc:654] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:22.158281 25955 tablet_bootstrap.cc:492] T 40223dbcbbc54f3c88d0902c7012d0d9 P 9d43cb0db93d43cda7a53e720c870cf2: Bootstrap starting.
I20260811 11:59:22.158864 25956 tablet_bootstrap.cc:492] T 40223dbcbbc54f3c88d0902c7012d0d9 P 27673094e2894a5fae1b560d43dc531a: Bootstrap starting.
I20260811 11:59:22.159036 25953 log.cc:824] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:22.160861 25955 tablet_bootstrap.cc:654] T 40223dbcbbc54f3c88d0902c7012d0d9 P 9d43cb0db93d43cda7a53e720c870cf2: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:22.161497 25953 tablet_bootstrap.cc:492] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6: No bootstrap required, opened a new log
I20260811 11:59:22.161628 25956 tablet_bootstrap.cc:654] T 40223dbcbbc54f3c88d0902c7012d0d9 P 27673094e2894a5fae1b560d43dc531a: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:22.161739 25953 ts_tablet_manager.cc:1397] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 11:59:22.162232 25955 log.cc:824] T 40223dbcbbc54f3c88d0902c7012d0d9 P 9d43cb0db93d43cda7a53e720c870cf2: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:22.162567 25956 log.cc:824] T 40223dbcbbc54f3c88d0902c7012d0d9 P 27673094e2894a5fae1b560d43dc531a: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:22.164026 25955 tablet_bootstrap.cc:492] T 40223dbcbbc54f3c88d0902c7012d0d9 P 9d43cb0db93d43cda7a53e720c870cf2: No bootstrap required, opened a new log
I20260811 11:59:22.164229 25955 ts_tablet_manager.cc:1397] T 40223dbcbbc54f3c88d0902c7012d0d9 P 9d43cb0db93d43cda7a53e720c870cf2: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 11:59:22.164645 25956 tablet_bootstrap.cc:492] T 40223dbcbbc54f3c88d0902c7012d0d9 P 27673094e2894a5fae1b560d43dc531a: No bootstrap required, opened a new log
I20260811 11:59:22.164849 25956 ts_tablet_manager.cc:1397] T 40223dbcbbc54f3c88d0902c7012d0d9 P 27673094e2894a5fae1b560d43dc531a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 11:59:22.166081 25953 raft_consensus.cc:348] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27673094e2894a5fae1b560d43dc531a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 46291 } } peers { permanent_uuid: "9d43cb0db93d43cda7a53e720c870cf2" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 35627 } } peers { permanent_uuid: "8da9337ee7fd4f6c8f0baea3444aabf6" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36319 } }
I20260811 11:59:22.166386 25953 raft_consensus.cc:374] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:22.166440 25953 raft_consensus.cc:729] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8da9337ee7fd4f6c8f0baea3444aabf6, State: Initialized, Role: FOLLOWER
I20260811 11:59:22.166998 25953 consensus_queue.cc:260] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27673094e2894a5fae1b560d43dc531a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 46291 } } peers { permanent_uuid: "9d43cb0db93d43cda7a53e720c870cf2" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 35627 } } peers { permanent_uuid: "8da9337ee7fd4f6c8f0baea3444aabf6" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36319 } }
I20260811 11:59:22.167316 25955 raft_consensus.cc:348] T 40223dbcbbc54f3c88d0902c7012d0d9 P 9d43cb0db93d43cda7a53e720c870cf2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27673094e2894a5fae1b560d43dc531a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 46291 } } peers { permanent_uuid: "9d43cb0db93d43cda7a53e720c870cf2" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 35627 } } peers { permanent_uuid: "8da9337ee7fd4f6c8f0baea3444aabf6" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36319 } }
I20260811 11:59:22.167574 25955 raft_consensus.cc:374] T 40223dbcbbc54f3c88d0902c7012d0d9 P 9d43cb0db93d43cda7a53e720c870cf2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:22.167639 25955 raft_consensus.cc:729] T 40223dbcbbc54f3c88d0902c7012d0d9 P 9d43cb0db93d43cda7a53e720c870cf2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9d43cb0db93d43cda7a53e720c870cf2, State: Initialized, Role: FOLLOWER
I20260811 11:59:22.168040 25932 heartbeater.cc:503] Master 127.6.143.190:45309 was elected leader, sending a full tablet report...
I20260811 11:59:22.168143 25953 ts_tablet_manager.cc:1428] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6: Time spent starting tablet: real 0.006s user 0.004s sys 0.004s
I20260811 11:59:22.169181 25955 consensus_queue.cc:260] T 40223dbcbbc54f3c88d0902c7012d0d9 P 9d43cb0db93d43cda7a53e720c870cf2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27673094e2894a5fae1b560d43dc531a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 46291 } } peers { permanent_uuid: "9d43cb0db93d43cda7a53e720c870cf2" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 35627 } } peers { permanent_uuid: "8da9337ee7fd4f6c8f0baea3444aabf6" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36319 } }
I20260811 11:59:22.169067 25956 raft_consensus.cc:348] T 40223dbcbbc54f3c88d0902c7012d0d9 P 27673094e2894a5fae1b560d43dc531a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27673094e2894a5fae1b560d43dc531a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 46291 } } peers { permanent_uuid: "9d43cb0db93d43cda7a53e720c870cf2" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 35627 } } peers { permanent_uuid: "8da9337ee7fd4f6c8f0baea3444aabf6" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36319 } }
I20260811 11:59:22.169373 25956 raft_consensus.cc:374] T 40223dbcbbc54f3c88d0902c7012d0d9 P 27673094e2894a5fae1b560d43dc531a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:22.169443 25956 raft_consensus.cc:729] T 40223dbcbbc54f3c88d0902c7012d0d9 P 27673094e2894a5fae1b560d43dc531a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27673094e2894a5fae1b560d43dc531a, State: Initialized, Role: FOLLOWER
I20260811 11:59:22.169858 25956 consensus_queue.cc:260] T 40223dbcbbc54f3c88d0902c7012d0d9 P 27673094e2894a5fae1b560d43dc531a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27673094e2894a5fae1b560d43dc531a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 46291 } } peers { permanent_uuid: "9d43cb0db93d43cda7a53e720c870cf2" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 35627 } } peers { permanent_uuid: "8da9337ee7fd4f6c8f0baea3444aabf6" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36319 } }
I20260811 11:59:22.170125 25955 ts_tablet_manager.cc:1428] T 40223dbcbbc54f3c88d0902c7012d0d9 P 9d43cb0db93d43cda7a53e720c870cf2: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260811 11:59:22.170574 25779 heartbeater.cc:503] Master 127.6.143.190:45309 was elected leader, sending a full tablet report...
I20260811 11:59:22.170936 25956 ts_tablet_manager.cc:1428] T 40223dbcbbc54f3c88d0902c7012d0d9 P 27673094e2894a5fae1b560d43dc531a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 11:59:22.171140 25628 heartbeater.cc:503] Master 127.6.143.190:45309 was elected leader, sending a full tablet report...
I20260811 11:59:22.306684 25960 raft_consensus.cc:482] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:59:22.306952 25960 raft_consensus.cc:504] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27673094e2894a5fae1b560d43dc531a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 46291 } } peers { permanent_uuid: "9d43cb0db93d43cda7a53e720c870cf2" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 35627 } } peers { permanent_uuid: "8da9337ee7fd4f6c8f0baea3444aabf6" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36319 } }
I20260811 11:59:22.308432 25960 leader_election.cc:288] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 27673094e2894a5fae1b560d43dc531a (127.6.143.130:46291), 9d43cb0db93d43cda7a53e720c870cf2 (127.6.143.129:35627)
I20260811 11:59:22.312780 25950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:22.308890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:55345 (local address 127.6.143.129:35627)
0811 11:59:22.309034 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:22.309038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:22.309053 (+ 15us) server_negotiation.cc:407] Connection header received
0811 11:59:22.309108 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:22.309110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:22.309155 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:22.309285 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:22.309646 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.310320 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.311569 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.312094 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.312192 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:22.312233 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:22.312549 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 11:59:22.312632 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 11:59:22.312793 25936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:22.308647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:35627 (local address 127.6.143.131:55345)
0811 11:59:22.308764 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:22.308781 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:22.308931 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:22.309298 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:22.309302 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:22.309324 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:22.309493 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:22.309503 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.310446 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.310449 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:22.311397 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:22.311407 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.312231 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.312236 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:22.312363 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:22.312418 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 11:59:22.312505 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 11:59:22.312604 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":17}
I20260811 11:59:22.312816 25948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:22.308855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:58801 (local address 127.6.143.130:46291)
0811 11:59:22.309020 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:22.309024 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:22.309476 (+ 452us) server_negotiation.cc:407] Connection header received
0811 11:59:22.309545 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:22.309552 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:22.309619 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:22.309700 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:22.310015 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.310657 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.311724 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.312233 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.312361 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:22.312420 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:22.312624 (+ 204us) server_negotiation.cc:299] Negotiation successful
0811 11:59:22.312700 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 11:59:22.312808 25963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:22.308647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:46291 (local address 127.6.143.131:58801)
0811 11:59:22.309366 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:22.309378 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:22.309467 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:22.309737 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:22.309740 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:22.309759 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:22.309878 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:22.309884 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.310775 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.310777 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:22.311584 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:22.311594 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.312354 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.312357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:22.312455 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:22.312499 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 11:59:22.312581 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 11:59:22.312642 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"mutex_wait_us":364,"thread_start_us":134,"threads_started":1}
I20260811 11:59:22.313617 25578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40223dbcbbc54f3c88d0902c7012d0d9" candidate_uuid: "8da9337ee7fd4f6c8f0baea3444aabf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d43cb0db93d43cda7a53e720c870cf2" is_pre_election: true
I20260811 11:59:22.313833 25727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40223dbcbbc54f3c88d0902c7012d0d9" candidate_uuid: "8da9337ee7fd4f6c8f0baea3444aabf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27673094e2894a5fae1b560d43dc531a" is_pre_election: true
I20260811 11:59:22.314023 25578 raft_consensus.cc:2454] T 40223dbcbbc54f3c88d0902c7012d0d9 P 9d43cb0db93d43cda7a53e720c870cf2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8da9337ee7fd4f6c8f0baea3444aabf6 in term 0.
I20260811 11:59:22.314168 25727 raft_consensus.cc:2454] T 40223dbcbbc54f3c88d0902c7012d0d9 P 27673094e2894a5fae1b560d43dc531a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8da9337ee7fd4f6c8f0baea3444aabf6 in term 0.
I20260811 11:59:22.314632 25804 leader_election.cc:302] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8da9337ee7fd4f6c8f0baea3444aabf6, 9d43cb0db93d43cda7a53e720c870cf2; no voters:
I20260811 11:59:22.314951 25960 raft_consensus.cc:2781] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:59:22.315048 25960 raft_consensus.cc:482] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:59:22.315107 25960 raft_consensus.cc:3037] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:22.316319 25960 raft_consensus.cc:504] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27673094e2894a5fae1b560d43dc531a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 46291 } } peers { permanent_uuid: "9d43cb0db93d43cda7a53e720c870cf2" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 35627 } } peers { permanent_uuid: "8da9337ee7fd4f6c8f0baea3444aabf6" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36319 } }
I20260811 11:59:22.316721 25960 leader_election.cc:288] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6 [CANDIDATE]: Term 1 election: Requested vote from peers 27673094e2894a5fae1b560d43dc531a (127.6.143.130:46291), 9d43cb0db93d43cda7a53e720c870cf2 (127.6.143.129:35627)
I20260811 11:59:22.317047 25727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40223dbcbbc54f3c88d0902c7012d0d9" candidate_uuid: "8da9337ee7fd4f6c8f0baea3444aabf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27673094e2894a5fae1b560d43dc531a"
I20260811 11:59:22.317143 25578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "40223dbcbbc54f3c88d0902c7012d0d9" candidate_uuid: "8da9337ee7fd4f6c8f0baea3444aabf6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9d43cb0db93d43cda7a53e720c870cf2"
I20260811 11:59:22.317181 25727 raft_consensus.cc:3037] T 40223dbcbbc54f3c88d0902c7012d0d9 P 27673094e2894a5fae1b560d43dc531a [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:22.317337 25578 raft_consensus.cc:3037] T 40223dbcbbc54f3c88d0902c7012d0d9 P 9d43cb0db93d43cda7a53e720c870cf2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:22.318300 25727 raft_consensus.cc:2454] T 40223dbcbbc54f3c88d0902c7012d0d9 P 27673094e2894a5fae1b560d43dc531a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da9337ee7fd4f6c8f0baea3444aabf6 in term 1.
I20260811 11:59:22.318611 25807 leader_election.cc:302] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27673094e2894a5fae1b560d43dc531a, 8da9337ee7fd4f6c8f0baea3444aabf6; no voters:
I20260811 11:59:22.318656 25578 raft_consensus.cc:2454] T 40223dbcbbc54f3c88d0902c7012d0d9 P 9d43cb0db93d43cda7a53e720c870cf2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8da9337ee7fd4f6c8f0baea3444aabf6 in term 1.
I20260811 11:59:22.318836 25960 raft_consensus.cc:2781] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:22.319116 25960 raft_consensus.cc:686] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6 [term 1 LEADER]: Becoming Leader. State: Replica: 8da9337ee7fd4f6c8f0baea3444aabf6, State: Running, Role: LEADER
I20260811 11:59:22.319443 25960 consensus_queue.cc:237] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27673094e2894a5fae1b560d43dc531a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 46291 } } peers { permanent_uuid: "9d43cb0db93d43cda7a53e720c870cf2" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 35627 } } peers { permanent_uuid: "8da9337ee7fd4f6c8f0baea3444aabf6" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36319 } }
I20260811 11:59:22.322775 25421 catalog_manager.cc:5495] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8da9337ee7fd4f6c8f0baea3444aabf6 (127.6.143.131). New cstate: current_term: 1 leader_uuid: "8da9337ee7fd4f6c8f0baea3444aabf6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27673094e2894a5fae1b560d43dc531a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 46291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9d43cb0db93d43cda7a53e720c870cf2" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 35627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8da9337ee7fd4f6c8f0baea3444aabf6" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36319 } health_report { overall_health: HEALTHY } } }
I20260811 11:59:22.338590 25952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:22.335075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40184 (local address 127.6.143.131:36319)
0811 11:59:22.335208 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:22.335211 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:22.335223 (+ 12us) server_negotiation.cc:407] Connection header received
0811 11:59:22.335356 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:22.335359 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:22.335405 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:22.335488 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:59:22.335707 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.336388 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.337236 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.337465 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.337552 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:22.337818 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:59:22.338215 (+ 397us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:59:22.338307 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:22.338371 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 11:59:22.338417 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 11:59:22.344379 25578 raft_consensus.cc:1261] T 40223dbcbbc54f3c88d0902c7012d0d9 P 9d43cb0db93d43cda7a53e720c870cf2 [term 1 FOLLOWER]: Refusing update from remote peer 8da9337ee7fd4f6c8f0baea3444aabf6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:59:22.344379 25727 raft_consensus.cc:1261] T 40223dbcbbc54f3c88d0902c7012d0d9 P 27673094e2894a5fae1b560d43dc531a [term 1 FOLLOWER]: Refusing update from remote peer 8da9337ee7fd4f6c8f0baea3444aabf6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260811 11:59:22.345157 25934 tablet.cc:2369] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:59:22.345118 25960 consensus_queue.cc:1035] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9d43cb0db93d43cda7a53e720c870cf2" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 35627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:59:22.345377 25964 consensus_queue.cc:1035] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27673094e2894a5fae1b560d43dc531a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 46291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:59:22.353261 25969 mvcc.cc:205] Tried to move back new op lower bound from 7317297407355379712 to 7317297407263502336. Current Snapshot: MvccSnapshot[applied={T|T < 7317297407355379712}]
I20260811 11:59:22.353472 25967 mvcc.cc:205] Tried to move back new op lower bound from 7317297407355379712 to 7317297407263502336. Current Snapshot: MvccSnapshot[applied={T|T < 7317297407355379712}]
I20260811 11:59:22.356732 25968 mvcc.cc:205] Tried to move back new op lower bound from 7317297407355379712 to 7317297407263502336. Current Snapshot: MvccSnapshot[applied={T|T < 7317297407355379712}]
I20260811 11:59:22.363409 25420 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:59:22.363571 25420 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55012:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:59:22.366037 25420 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6dc48d0de64544c28eecf257c552f882: Sending DeleteTablet for 3 replicas of tablet 40223dbcbbc54f3c88d0902c7012d0d9
I20260811 11:59:22.366779 25704 tablet_service.cc:1514] Processing DeleteTablet for tablet 40223dbcbbc54f3c88d0902c7012d0d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:59:22 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:38004
I20260811 11:59:22.366850 25556 tablet_service.cc:1514] Processing DeleteTablet for tablet 40223dbcbbc54f3c88d0902c7012d0d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:59:22 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:55396
I20260811 11:59:22.366824 25859 tablet_service.cc:1514] Processing DeleteTablet for tablet 40223dbcbbc54f3c88d0902c7012d0d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:59:22 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:40182
I20260811 11:59:22.367295 25978 tablet_replica.cc:330] T 40223dbcbbc54f3c88d0902c7012d0d9 P 9d43cb0db93d43cda7a53e720c870cf2: stopping tablet replica
I20260811 11:59:22.367436 25976 tablet_replica.cc:330] T 40223dbcbbc54f3c88d0902c7012d0d9 P 27673094e2894a5fae1b560d43dc531a: stopping tablet replica
I20260811 11:59:22.367645 25977 tablet_replica.cc:330] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6: stopping tablet replica
I20260811 11:59:22.367940 25976 raft_consensus.cc:2229] T 40223dbcbbc54f3c88d0902c7012d0d9 P 27673094e2894a5fae1b560d43dc531a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:59:22.368009 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 25483
I20260811 11:59:22.368006 25977 raft_consensus.cc:2229] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6 [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:59:22.368356 25976 raft_consensus.cc:2258] T 40223dbcbbc54f3c88d0902c7012d0d9 P 27673094e2894a5fae1b560d43dc531a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:59:22.368387 25977 raft_consensus.cc:2258] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:59:22.370611 25976 ts_tablet_manager.cc:1905] T 40223dbcbbc54f3c88d0902c7012d0d9 P 27673094e2894a5fae1b560d43dc531a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:59:22.370615 25977 ts_tablet_manager.cc:1905] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:59:22.373785 25976 ts_tablet_manager.cc:1918] T 40223dbcbbc54f3c88d0902c7012d0d9 P 27673094e2894a5fae1b560d43dc531a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:59:22.373944 25976 log.cc:1196] T 40223dbcbbc54f3c88d0902c7012d0d9 P 27673094e2894a5fae1b560d43dc531a: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-1/wal/wals/40223dbcbbc54f3c88d0902c7012d0d9
I20260811 11:59:22.374331 25976 ts_tablet_manager.cc:1939] T 40223dbcbbc54f3c88d0902c7012d0d9 P 27673094e2894a5fae1b560d43dc531a: Deleting consensus metadata
I20260811 11:59:22.374534 25977 ts_tablet_manager.cc:1918] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:59:22.374639 25977 log.cc:1196] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786449319379472-6718-0/minicluster-data/ts-2/wal/wals/40223dbcbbc54f3c88d0902c7012d0d9
I20260811 11:59:22.374977 25977 ts_tablet_manager.cc:1939] T 40223dbcbbc54f3c88d0902c7012d0d9 P 8da9337ee7fd4f6c8f0baea3444aabf6: Deleting consensus metadata
I20260811 11:59:22.375788 25405 catalog_manager.cc:4841] TS 27673094e2894a5fae1b560d43dc531a (127.6.143.130:46291): tablet 40223dbcbbc54f3c88d0902c7012d0d9 (table test-table [id=9d74f0d2d258493fbad39d849b367577]) successfully deleted
I20260811 11:59:22.376132 25401 catalog_manager.cc:4841] TS 8da9337ee7fd4f6c8f0baea3444aabf6 (127.6.143.131:36319): tablet 40223dbcbbc54f3c88d0902c7012d0d9 (table test-table [id=9d74f0d2d258493fbad39d849b367577]) successfully deleted
W20260811 11:59:22.380156 25401 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.143.129:35627 (error 108)
I20260811 11:59:22.380312 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 25635
I20260811 11:59:22.382982 25951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:22.382623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:35627 (local address 127.0.0.1:55408)
0811 11:59:22.382807 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:22.382881 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.129:35627: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
W20260811 11:59:22.383220 25401 catalog_manager.cc:4584] TS 9d43cb0db93d43cda7a53e720c870cf2 (127.6.143.129:35627): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 40223dbcbbc54f3c88d0902c7012d0d9: Network error: Client connection negotiation failed: client connection to 127.6.143.129:35627: connect: Connection refused (error 111)
I20260811 11:59:22.389986 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 25784
I20260811 11:59:22.396083 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 25385
2026-08-11T11:59:22Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2492 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T11:59:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:59:22Z Disabled control of system clock
I20260811 11:59:22.429239 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:37107
--webserver_interface=127.6.143.190
--webserver_port=0
--builtin_ntp_servers=127.6.143.148:34547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:37107
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 11:59:22.539057 25983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:22.539294 25983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:22.539326 25983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:22.543138 25983 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:59:22.543182 25983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:22.543197 25983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:22.543213 25983 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:59:22.543228 25983 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:59:22.547705 25983 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:34547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:37107
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:37107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:22.548703 25983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:22.550000 25983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:22.558406 25989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:22.558568 25983 server_base.cc:1034] running on GCE node
W20260811 11:59:22.558429 25992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:22.558429 25990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:22.559314 25983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:22.560559 25983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:22.561777 25983 hybrid_clock.cc:648] HybridClock initialized: now 1786449562561755 us; error 42 us; skew 500 ppm
I20260811 11:59:22.563697 25983 webserver.cc:460] Webserver started at http://127.6.143.190:39481/ using document root <none> and password file <none>
I20260811 11:59:22.564419 25983 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:22.564530 25983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:22.564793 25983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:22.566991 25983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "059517940edd436ea975b86d34bce457"
format_stamp: "Formatted at 2026-08-11 11:59:22 on dist-test-slave-zkpd"
server_key: "1ddbdc8479e4078d8e73ba3581d83fe5"
server_key_iv: "75d8e769822efef62bcf40575f9fa596"
server_key_version: "encryptionkey@0"
I20260811 11:59:22.567607 25983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "059517940edd436ea975b86d34bce457"
format_stamp: "Formatted at 2026-08-11 11:59:22 on dist-test-slave-zkpd"
server_key: "1ddbdc8479e4078d8e73ba3581d83fe5"
server_key_iv: "75d8e769822efef62bcf40575f9fa596"
server_key_version: "encryptionkey@0"
I20260811 11:59:22.571614 25983 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 11:59:22.574190 25999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:22.575461 25983 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 11:59:22.575616 25983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "059517940edd436ea975b86d34bce457"
format_stamp: "Formatted at 2026-08-11 11:59:22 on dist-test-slave-zkpd"
server_key: "1ddbdc8479e4078d8e73ba3581d83fe5"
server_key_iv: "75d8e769822efef62bcf40575f9fa596"
server_key_version: "encryptionkey@0"
I20260811 11:59:22.575738 25983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:22.593020 25983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:22.593715 25983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:22.593928 25983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:22.602749 25983 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:37107
I20260811 11:59:22.602746 26060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:37107 every 8 connection(s)
I20260811 11:59:22.603807 25983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:59:22.604993 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 25983
I20260811 11:59:22.605129 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:59:22.605450 6718 external_mini_cluster.cc:1428] Setting key 37f1f6ae53ce2da7a459901fabf215cf
I20260811 11:59:22.607733 26061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:22.613343 26061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 059517940edd436ea975b86d34bce457: Bootstrap starting.
I20260811 11:59:22.614913 26065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:22.606887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35966 (local address 127.6.143.190:37107)
0811 11:59:22.607572 (+ 685us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:22.607587 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:22.607622 (+ 35us) server_negotiation.cc:407] Connection header received
0811 11:59:22.608396 (+ 774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:22.608415 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:22.608817 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:22.609188 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:22.609667 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.610635 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.611561 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.612119 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.612298 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:22.613190 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:22.613195 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:22.613207 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:22.613631 (+ 424us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:22.613711 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:22.613733 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:22.613843 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:22.613980 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 11:59:22.614227 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":151,"threads_started":1}
I20260811 11:59:22.615634 26061 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 059517940edd436ea975b86d34bce457: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:22.616581 26061 log.cc:824] T 00000000000000000000000000000000 P 059517940edd436ea975b86d34bce457: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:22.618372 26061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 059517940edd436ea975b86d34bce457: No bootstrap required, opened a new log
I20260811 11:59:22.621786 26061 raft_consensus.cc:348] T 00000000000000000000000000000000 P 059517940edd436ea975b86d34bce457 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "059517940edd436ea975b86d34bce457" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 37107 } }
I20260811 11:59:22.622001 26061 raft_consensus.cc:374] T 00000000000000000000000000000000 P 059517940edd436ea975b86d34bce457 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:22.622035 26061 raft_consensus.cc:729] T 00000000000000000000000000000000 P 059517940edd436ea975b86d34bce457 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 059517940edd436ea975b86d34bce457, State: Initialized, Role: FOLLOWER
I20260811 11:59:22.622457 26061 consensus_queue.cc:260] T 00000000000000000000000000000000 P 059517940edd436ea975b86d34bce457 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "059517940edd436ea975b86d34bce457" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 37107 } }
I20260811 11:59:22.622605 26061 raft_consensus.cc:388] T 00000000000000000000000000000000 P 059517940edd436ea975b86d34bce457 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:59:22.622674 26061 raft_consensus.cc:482] T 00000000000000000000000000000000 P 059517940edd436ea975b86d34bce457 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:59:22.622772 26061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 059517940edd436ea975b86d34bce457 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:22.623796 26061 raft_consensus.cc:504] T 00000000000000000000000000000000 P 059517940edd436ea975b86d34bce457 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "059517940edd436ea975b86d34bce457" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 37107 } }
I20260811 11:59:22.624161 26061 leader_election.cc:302] T 00000000000000000000000000000000 P 059517940edd436ea975b86d34bce457 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 059517940edd436ea975b86d34bce457; no voters:
I20260811 11:59:22.624455 26061 leader_election.cc:288] T 00000000000000000000000000000000 P 059517940edd436ea975b86d34bce457 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:59:22.624608 26067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 059517940edd436ea975b86d34bce457 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:22.624873 26067 raft_consensus.cc:686] T 00000000000000000000000000000000 P 059517940edd436ea975b86d34bce457 [term 1 LEADER]: Becoming Leader. State: Replica: 059517940edd436ea975b86d34bce457, State: Running, Role: LEADER
I20260811 11:59:22.625187 26067 consensus_queue.cc:237] T 00000000000000000000000000000000 P 059517940edd436ea975b86d34bce457 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "059517940edd436ea975b86d34bce457" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 37107 } }
I20260811 11:59:22.625746 26061 sys_catalog.cc:564] T 00000000000000000000000000000000 P 059517940edd436ea975b86d34bce457 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:59:22.626971 26068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 059517940edd436ea975b86d34bce457 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "059517940edd436ea975b86d34bce457" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "059517940edd436ea975b86d34bce457" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 37107 } } }
I20260811 11:59:22.626991 26069 sys_catalog.cc:455] T 00000000000000000000000000000000 P 059517940edd436ea975b86d34bce457 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 059517940edd436ea975b86d34bce457. Latest consensus state: current_term: 1 leader_uuid: "059517940edd436ea975b86d34bce457" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "059517940edd436ea975b86d34bce457" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 37107 } } }
I20260811 11:59:22.627110 26068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 059517940edd436ea975b86d34bce457 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:22.627132 26069 sys_catalog.cc:458] T 00000000000000000000000000000000 P 059517940edd436ea975b86d34bce457 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:22.627467 26076 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:59:22.630569 26076 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:59:22.636477 26076 catalog_manager.cc:1347] Generated new cluster ID: cedf5d83a9df4ddeb7cebe59d262c1b1
I20260811 11:59:22.636545 26076 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:59:22.649818 26076 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:59:22.650696 26076 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:59:22.666585 26076 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 059517940edd436ea975b86d34bce457: Generated new TSK 0
I20260811 11:59:22.667377 26076 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:59:22.677177 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:0
--local_ip_for_outbound_sockets=127.6.143.129
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:37107
--builtin_ntp_servers=127.6.143.148:34547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 11:59:22.791499 26090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:22.791796 26090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:22.791879 26090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:22.795917 26090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:22.795993 26090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:22.796082 26090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:59:22.800917 26090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:34547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:37107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:22.801970 26090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:22.803363 26090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:22.812934 26100 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:22.812987 26096 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:22.812961 26095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:22.813287 26090 server_base.cc:1034] running on GCE node
I20260811 11:59:22.813737 26090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:22.814366 26090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:22.815534 26090 hybrid_clock.cc:648] HybridClock initialized: now 1786449562815519 us; error 26 us; skew 500 ppm
I20260811 11:59:22.817478 26090 webserver.cc:460] Webserver started at http://127.6.143.129:44851/ using document root <none> and password file <none>
I20260811 11:59:22.818089 26090 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:22.818156 26090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:22.818439 26090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:22.820623 26090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-0/data/instance:
uuid: "24cf6f1f898c42f48bd004c5591a2662"
format_stamp: "Formatted at 2026-08-11 11:59:22 on dist-test-slave-zkpd"
server_key: "c67ebb332b1bf9a34897c545bc5c53dd"
server_key_iv: "44c99607b710f70295f0f22074e41fe4"
server_key_version: "encryptionkey@0"
I20260811 11:59:22.821183 26090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance:
uuid: "24cf6f1f898c42f48bd004c5591a2662"
format_stamp: "Formatted at 2026-08-11 11:59:22 on dist-test-slave-zkpd"
server_key: "c67ebb332b1bf9a34897c545bc5c53dd"
server_key_iv: "44c99607b710f70295f0f22074e41fe4"
server_key_version: "encryptionkey@0"
I20260811 11:59:22.824971 26090 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 11:59:22.827579 26107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:22.828954 26090 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 11:59:22.829079 26090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "24cf6f1f898c42f48bd004c5591a2662"
format_stamp: "Formatted at 2026-08-11 11:59:22 on dist-test-slave-zkpd"
server_key: "c67ebb332b1bf9a34897c545bc5c53dd"
server_key_iv: "44c99607b710f70295f0f22074e41fe4"
server_key_version: "encryptionkey@0"
I20260811 11:59:22.829185 26090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:22.855330 26090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:22.856177 26090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:22.856371 26090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:22.856976 26090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:59:22.858012 26090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:59:22.858062 26090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:22.858132 26090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:59:22.858165 26090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:22.867451 26090 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:33427
I20260811 11:59:22.867527 26238 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:33427 every 8 connection(s)
I20260811 11:59:22.868455 26090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:59:22.874876 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 26090
I20260811 11:59:22.875025 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance
I20260811 11:59:22.875350 6718 external_mini_cluster.cc:1428] Setting key ec5491190131d38962bdef6f967679f7
I20260811 11:59:22.877449 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:0
--local_ip_for_outbound_sockets=127.6.143.130
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:37107
--builtin_ntp_servers=127.6.143.148:34547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 11:59:22.878258 26065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:22.870307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:47589 (local address 127.6.143.190:37107)
0811 11:59:22.870473 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:22.870478 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:22.871552 (+ 1074us) server_negotiation.cc:407] Connection header received
0811 11:59:22.872964 (+ 1412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:22.872967 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:22.873016 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:22.873093 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:22.873979 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.874509 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.876149 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.876390 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.876514 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:22.877761 (+ 1247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:22.877764 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:22.877766 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:22.877779 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:22.877798 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:22.877805 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:22.877868 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:22.878064 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 11:59:22.878132 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":64}
I20260811 11:59:22.879463 26241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:22.870718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:37107 (local address 127.6.143.129:47589)
0811 11:59:22.871377 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:22.871427 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:22.872767 (+ 1340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:22.873263 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:22.873269 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:22.873352 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:59:22.873851 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:22.873865 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.874666 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.874671 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:22.876016 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:22.876027 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:22.876555 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:22.876559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:22.876792 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:22.877441 (+ 649us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:59:22.877443 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:59:22.877500 (+ 57us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:59:22.877591 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:59:22.877900 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:59:22.877911 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:59:22.877920 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 11:59:22.878053 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 11:59:22.878359 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":119,"threads_started":1}
I20260811 11:59:22.880887 26239 heartbeater.cc:347] Connected to a master server at 127.6.143.190:37107
I20260811 11:59:22.881134 26239 heartbeater.cc:464] Registering TS with master...
I20260811 11:59:22.881657 26239 heartbeater.cc:511] Master 127.6.143.190:37107 requested a full tablet report, sending...
I20260811 11:59:22.883425 26021 ts_manager.cc:194] Registered new tserver with Master: 24cf6f1f898c42f48bd004c5591a2662 (127.6.143.129:33427)
I20260811 11:59:22.885231 26021 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.143.129:47589
W20260811 11:59:22.991219 26242 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:22.991451 26242 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:22.991487 26242 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:22.995436 26242 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:22.995487 26242 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:22.995537 26242 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:59:23.000232 26242 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:34547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:37107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26242
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:23.001149 26242 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:23.002440 26242 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:23.011745 26247 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:23.011907 26250 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:23.011745 26248 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:23.012072 26242 server_base.cc:1034] running on GCE node
I20260811 11:59:23.012481 26242 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:23.013108 26242 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:23.014285 26242 hybrid_clock.cc:648] HybridClock initialized: now 1786449563014266 us; error 30 us; skew 500 ppm
I20260811 11:59:23.016201 26242 webserver.cc:460] Webserver started at http://127.6.143.130:40377/ using document root <none> and password file <none>
I20260811 11:59:23.016788 26242 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:23.016850 26242 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:23.017105 26242 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:23.019131 26242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-1/data/instance:
uuid: "87e4ddbef7b846fba1f05fa8fa5ae6c5"
format_stamp: "Formatted at 2026-08-11 11:59:23 on dist-test-slave-zkpd"
server_key: "f70837920f347a2f1b4dee594fd43541"
server_key_iv: "52737db3681d9c8f438b12f1613e1c70"
server_key_version: "encryptionkey@0"
I20260811 11:59:23.019743 26242 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance:
uuid: "87e4ddbef7b846fba1f05fa8fa5ae6c5"
format_stamp: "Formatted at 2026-08-11 11:59:23 on dist-test-slave-zkpd"
server_key: "f70837920f347a2f1b4dee594fd43541"
server_key_iv: "52737db3681d9c8f438b12f1613e1c70"
server_key_version: "encryptionkey@0"
I20260811 11:59:23.023463 26242 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 11:59:23.025969 26257 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:23.027268 26242 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:59:23.027407 26242 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "87e4ddbef7b846fba1f05fa8fa5ae6c5"
format_stamp: "Formatted at 2026-08-11 11:59:23 on dist-test-slave-zkpd"
server_key: "f70837920f347a2f1b4dee594fd43541"
server_key_iv: "52737db3681d9c8f438b12f1613e1c70"
server_key_version: "encryptionkey@0"
I20260811 11:59:23.027521 26242 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:23.050897 26242 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:23.051601 26242 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:23.051811 26242 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:23.052443 26242 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:59:23.053423 26242 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:59:23.053500 26242 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:23.053570 26242 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:59:23.053622 26242 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:23.062338 26242 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:43385
I20260811 11:59:23.062367 26381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:43385 every 8 connection(s)
I20260811 11:59:23.063201 26242 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
I20260811 11:59:23.065507 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 26242
I20260811 11:59:23.065613 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance
I20260811 11:59:23.065982 6718 external_mini_cluster.cc:1428] Setting key dd221db8251e50053167c47365fe1f6b
I20260811 11:59:23.068228 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:0
--local_ip_for_outbound_sockets=127.6.143.131
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:37107
--builtin_ntp_servers=127.6.143.148:34547
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 11:59:23.073782 26065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:23.065059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:51749 (local address 127.6.143.190:37107)
0811 11:59:23.065198 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:23.065201 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:23.066422 (+ 1221us) server_negotiation.cc:407] Connection header received
0811 11:59:23.068212 (+ 1790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:23.068216 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:23.068262 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:23.068397 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:23.069648 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.070367 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.071504 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.071763 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.071850 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:23.073218 (+ 1368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:23.073221 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:23.073223 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:23.073234 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:23.073252 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:23.073261 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:23.073322 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:23.073529 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 11:59:23.073578 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 11:59:23.074966 26385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:23.065388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:37107 (local address 127.6.143.130:51749)
0811 11:59:23.066259 (+ 871us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:23.066313 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:23.067958 (+ 1645us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:23.068607 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:23.068619 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:23.068727 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:59:23.069440 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:23.069463 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.070515 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.070518 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:23.071382 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:23.071391 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.071989 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.071997 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:23.072232 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:23.072877 (+ 645us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:59:23.072882 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:59:23.072941 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:59:23.073047 (+ 106us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:59:23.073356 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:59:23.073371 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:59:23.073381 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 11:59:23.073535 (+ 154us) client_negotiation.cc:240] Negotiation successful
0811 11:59:23.073876 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":187,"threads_started":1}
I20260811 11:59:23.076186 26382 heartbeater.cc:347] Connected to a master server at 127.6.143.190:37107
I20260811 11:59:23.076439 26382 heartbeater.cc:464] Registering TS with master...
I20260811 11:59:23.076946 26382 heartbeater.cc:511] Master 127.6.143.190:37107 requested a full tablet report, sending...
I20260811 11:59:23.077971 26021 ts_manager.cc:194] Registered new tserver with Master: 87e4ddbef7b846fba1f05fa8fa5ae6c5 (127.6.143.130:43385)
I20260811 11:59:23.078534 26021 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.143.130:51749
W20260811 11:59:23.184381 26386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:23.184628 26386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:23.184700 26386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:23.188673 26386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:23.188722 26386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:23.188814 26386 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:59:23.193470 26386 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:34547
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:37107
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:23.194568 26386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:23.195976 26386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:23.205425 26394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:23.205427 26392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:23.205577 26386 server_base.cc:1034] running on GCE node
W20260811 11:59:23.205646 26391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:23.206157 26386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:23.206727 26386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:23.207926 26386 hybrid_clock.cc:648] HybridClock initialized: now 1786449563207862 us; error 42 us; skew 500 ppm
I20260811 11:59:23.209770 26386 webserver.cc:460] Webserver started at http://127.6.143.131:41663/ using document root <none> and password file <none>
I20260811 11:59:23.210348 26386 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:23.210419 26386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:23.210584 26386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:23.212615 26386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-2/data/instance:
uuid: "51284b98c44a41f9b321cf3e93a9f0ad"
format_stamp: "Formatted at 2026-08-11 11:59:23 on dist-test-slave-zkpd"
server_key: "42bac4388d42bd752ea76ea84e84a344"
server_key_iv: "d66c48760d1108ea83e7165d885aa61a"
server_key_version: "encryptionkey@0"
I20260811 11:59:23.213125 26386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance:
uuid: "51284b98c44a41f9b321cf3e93a9f0ad"
format_stamp: "Formatted at 2026-08-11 11:59:23 on dist-test-slave-zkpd"
server_key: "42bac4388d42bd752ea76ea84e84a344"
server_key_iv: "d66c48760d1108ea83e7165d885aa61a"
server_key_version: "encryptionkey@0"
I20260811 11:59:23.216761 26386 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 11:59:23.219142 26400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:23.220425 26386 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:59:23.220561 26386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "51284b98c44a41f9b321cf3e93a9f0ad"
format_stamp: "Formatted at 2026-08-11 11:59:23 on dist-test-slave-zkpd"
server_key: "42bac4388d42bd752ea76ea84e84a344"
server_key_iv: "d66c48760d1108ea83e7165d885aa61a"
server_key_version: "encryptionkey@0"
I20260811 11:59:23.220664 26386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:23.237099 26386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:23.237767 26386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:23.237984 26386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:23.238587 26386 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:59:23.239549 26386 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:59:23.239598 26386 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:23.239667 26386 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:59:23.239701 26386 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:23.248944 26386 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:35581
I20260811 11:59:23.248963 26522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:35581 every 8 connection(s)
I20260811 11:59:23.249828 26386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:59:23.256287 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 26386
I20260811 11:59:23.256378 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance
I20260811 11:59:23.256734 6718 external_mini_cluster.cc:1428] Setting key 6890ee12a768975f048d448264ae896e
I20260811 11:59:23.259318 26065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:23.251666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:51377 (local address 127.6.143.190:37107)
0811 11:59:23.251814 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:23.251818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:23.252917 (+ 1099us) server_negotiation.cc:407] Connection header received
0811 11:59:23.254287 (+ 1370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:23.254289 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:23.254336 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:23.254417 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:23.255259 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.255779 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.257144 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.257458 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.257543 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:23.258800 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:23.258804 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:23.258806 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:23.258820 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:23.258841 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:23.258853 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:23.258954 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:23.259122 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 11:59:23.259182 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 11:59:23.260210 26526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:23.252103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:37107 (local address 127.6.143.131:51377)
0811 11:59:23.252752 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:23.252791 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:23.254086 (+ 1295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:23.254582 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:23.254588 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:23.254669 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:59:23.255134 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:23.255145 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.255969 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.255972 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:23.256996 (+ 1024us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:23.257007 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.257628 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.257633 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:23.257858 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:23.258540 (+ 682us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:59:23.258542 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:59:23.258581 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:59:23.258643 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:59:23.258969 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:59:23.258977 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:59:23.258981 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:59:23.259115 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 11:59:23.259389 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":111,"threads_started":1}
I20260811 11:59:23.261237 26523 heartbeater.cc:347] Connected to a master server at 127.6.143.190:37107
I20260811 11:59:23.261472 26523 heartbeater.cc:464] Registering TS with master...
I20260811 11:59:23.261981 26523 heartbeater.cc:511] Master 127.6.143.190:37107 requested a full tablet report, sending...
I20260811 11:59:23.262935 26021 ts_manager.cc:194] Registered new tserver with Master: 51284b98c44a41f9b321cf3e93a9f0ad (127.6.143.131:35581)
I20260811 11:59:23.263504 26021 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.143.131:51377
I20260811 11:59:23.270870 6718 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:59:23.276381 26065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:23.272881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35968 (local address 127.6.143.190:37107)
0811 11:59:23.273022 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:23.273026 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:23.273198 (+ 172us) server_negotiation.cc:407] Connection header received
0811 11:59:23.273295 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:23.273297 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:23.273333 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:23.273401 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:23.273648 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.274145 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.274926 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.275131 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.275207 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:23.276020 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:23.276035 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:23.276037 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:23.276048 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:23.276066 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:23.276073 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:23.276136 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:23.276240 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 11:59:23.276286 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 11:59:23.279147 26021 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35968:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:59:23.282285 26021 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:59:23.297397 26538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:23.291631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34270 (local address 127.6.143.131:35581)
0811 11:59:23.292906 (+ 1275us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:23.292919 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:23.292944 (+ 25us) server_negotiation.cc:407] Connection header received
0811 11:59:23.293025 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:23.293030 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:23.293204 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:23.293364 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:23.293606 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.294714 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.296102 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.296639 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.296773 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:23.296980 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:23.297098 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 11:59:23.297212 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1151,"thread_start_us":181,"threads_started":1}
I20260811 11:59:23.297399 26535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:23.290903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:35581 (local address 127.0.0.1:34270)
0811 11:59:23.291245 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:23.291290 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:23.291434 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:23.293356 (+ 1922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:23.293359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:23.293378 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:23.293474 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:23.293481 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.294835 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.294838 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:23.295990 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:23.295998 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.296775 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.296778 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:23.296884 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:23.296933 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 11:59:23.297066 (+ 133us) client_negotiation.cc:240] Negotiation successful
0811 11:59:23.297220 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":41,"thread_start_us":96,"threads_started":1}
I20260811 11:59:23.297451 26537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:23.290970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:33427 (local address 127.0.0.1:51822)
0811 11:59:23.291815 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:23.291831 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:23.291944 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:23.292659 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:23.292665 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:23.292689 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:23.292816 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:23.292822 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.294351 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.294354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:23.295805 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:23.295817 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.296832 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.296836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:23.296947 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:23.296971 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 11:59:23.297053 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 11:59:23.297224 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":777,"mutex_wait_us":54,"thread_start_us":108,"threads_started":1}
I20260811 11:59:23.297638 26536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:23.291240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:43385 (local address 127.0.0.1:59646)
0811 11:59:23.292138 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:23.292151 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:23.292236 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:23.292988 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:23.292991 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:23.293008 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:23.293100 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:23.293105 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.294258 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.294264 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:23.295694 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:23.295705 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.296819 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.296823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:23.297337 (+ 514us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:23.297361 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 11:59:23.297434 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 11:59:23.297507 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":841,"mutex_wait_us":706,"thread_start_us":92,"threads_started":1}
I20260811 11:59:23.298473 26540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:23.292027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59646 (local address 127.6.143.130:43385)
0811 11:59:23.292510 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:23.292520 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:23.292541 (+ 21us) server_negotiation.cc:407] Connection header received
0811 11:59:23.292636 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:23.292641 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:23.292831 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:23.293022 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:23.293220 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.294140 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.295826 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.296668 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.296925 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:23.297135 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:23.298183 (+ 1048us) server_negotiation.cc:299] Negotiation successful
0811 11:59:23.298286 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":95,"threads_started":1}
I20260811 11:59:23.298560 26539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:23.291727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51822 (local address 127.6.143.129:33427)
0811 11:59:23.292177 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:23.292189 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:23.292214 (+ 25us) server_negotiation.cc:407] Connection header received
0811 11:59:23.292296 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:23.292301 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:23.292500 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:23.292630 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:23.292964 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.294184 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.295938 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.296668 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.297799 (+ 1131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:23.298014 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:23.298068 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 11:59:23.298171 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":98,"threads_started":1}
I20260811 11:59:23.301173 26453 tablet_service.cc:1467] Processing CreateTablet for tablet 73dcf39593364c67ac6f1670a34ff018 (DEFAULT_TABLE table=test-table [id=062c37e7bb9945738f44bb34ab5d7f3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:59:23.301344 26312 tablet_service.cc:1467] Processing CreateTablet for tablet 73dcf39593364c67ac6f1670a34ff018 (DEFAULT_TABLE table=test-table [id=062c37e7bb9945738f44bb34ab5d7f3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:59:23.301681 26166 tablet_service.cc:1467] Processing CreateTablet for tablet 73dcf39593364c67ac6f1670a34ff018 (DEFAULT_TABLE table=test-table [id=062c37e7bb9945738f44bb34ab5d7f3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:59:23.302357 26453 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73dcf39593364c67ac6f1670a34ff018. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:23.302428 26312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73dcf39593364c67ac6f1670a34ff018. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:23.302587 26166 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73dcf39593364c67ac6f1670a34ff018. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:23.308689 26541 tablet_bootstrap.cc:492] T 73dcf39593364c67ac6f1670a34ff018 P 51284b98c44a41f9b321cf3e93a9f0ad: Bootstrap starting.
I20260811 11:59:23.310786 26543 tablet_bootstrap.cc:492] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5: Bootstrap starting.
I20260811 11:59:23.311143 26541 tablet_bootstrap.cc:654] T 73dcf39593364c67ac6f1670a34ff018 P 51284b98c44a41f9b321cf3e93a9f0ad: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:23.312007 26541 log.cc:824] T 73dcf39593364c67ac6f1670a34ff018 P 51284b98c44a41f9b321cf3e93a9f0ad: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:23.312248 26542 tablet_bootstrap.cc:492] T 73dcf39593364c67ac6f1670a34ff018 P 24cf6f1f898c42f48bd004c5591a2662: Bootstrap starting.
I20260811 11:59:23.312801 26543 tablet_bootstrap.cc:654] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:23.313642 26543 log.cc:824] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:23.314116 26541 tablet_bootstrap.cc:492] T 73dcf39593364c67ac6f1670a34ff018 P 51284b98c44a41f9b321cf3e93a9f0ad: No bootstrap required, opened a new log
I20260811 11:59:23.314332 26541 ts_tablet_manager.cc:1397] T 73dcf39593364c67ac6f1670a34ff018 P 51284b98c44a41f9b321cf3e93a9f0ad: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 11:59:23.314450 26542 tablet_bootstrap.cc:654] T 73dcf39593364c67ac6f1670a34ff018 P 24cf6f1f898c42f48bd004c5591a2662: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:23.315277 26542 log.cc:824] T 73dcf39593364c67ac6f1670a34ff018 P 24cf6f1f898c42f48bd004c5591a2662: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:23.315848 26543 tablet_bootstrap.cc:492] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5: No bootstrap required, opened a new log
I20260811 11:59:23.316130 26543 ts_tablet_manager.cc:1397] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260811 11:59:23.317427 26542 tablet_bootstrap.cc:492] T 73dcf39593364c67ac6f1670a34ff018 P 24cf6f1f898c42f48bd004c5591a2662: No bootstrap required, opened a new log
I20260811 11:59:23.317623 26542 ts_tablet_manager.cc:1397] T 73dcf39593364c67ac6f1670a34ff018 P 24cf6f1f898c42f48bd004c5591a2662: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 11:59:23.317883 26541 raft_consensus.cc:348] T 73dcf39593364c67ac6f1670a34ff018 P 51284b98c44a41f9b321cf3e93a9f0ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51284b98c44a41f9b321cf3e93a9f0ad" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 35581 } } peers { permanent_uuid: "24cf6f1f898c42f48bd004c5591a2662" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 33427 } } peers { permanent_uuid: "87e4ddbef7b846fba1f05fa8fa5ae6c5" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43385 } }
I20260811 11:59:23.318123 26541 raft_consensus.cc:374] T 73dcf39593364c67ac6f1670a34ff018 P 51284b98c44a41f9b321cf3e93a9f0ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:23.318185 26541 raft_consensus.cc:729] T 73dcf39593364c67ac6f1670a34ff018 P 51284b98c44a41f9b321cf3e93a9f0ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51284b98c44a41f9b321cf3e93a9f0ad, State: Initialized, Role: FOLLOWER
I20260811 11:59:23.318642 26541 consensus_queue.cc:260] T 73dcf39593364c67ac6f1670a34ff018 P 51284b98c44a41f9b321cf3e93a9f0ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51284b98c44a41f9b321cf3e93a9f0ad" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 35581 } } peers { permanent_uuid: "24cf6f1f898c42f48bd004c5591a2662" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 33427 } } peers { permanent_uuid: "87e4ddbef7b846fba1f05fa8fa5ae6c5" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43385 } }
I20260811 11:59:23.319736 26541 ts_tablet_manager.cc:1428] T 73dcf39593364c67ac6f1670a34ff018 P 51284b98c44a41f9b321cf3e93a9f0ad: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 11:59:23.320211 26523 heartbeater.cc:503] Master 127.6.143.190:37107 was elected leader, sending a full tablet report...
I20260811 11:59:23.320526 26543 raft_consensus.cc:348] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51284b98c44a41f9b321cf3e93a9f0ad" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 35581 } } peers { permanent_uuid: "24cf6f1f898c42f48bd004c5591a2662" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 33427 } } peers { permanent_uuid: "87e4ddbef7b846fba1f05fa8fa5ae6c5" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43385 } }
I20260811 11:59:23.320792 26543 raft_consensus.cc:374] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:23.320853 26543 raft_consensus.cc:729] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87e4ddbef7b846fba1f05fa8fa5ae6c5, State: Initialized, Role: FOLLOWER
I20260811 11:59:23.321313 26543 consensus_queue.cc:260] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51284b98c44a41f9b321cf3e93a9f0ad" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 35581 } } peers { permanent_uuid: "24cf6f1f898c42f48bd004c5591a2662" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 33427 } } peers { permanent_uuid: "87e4ddbef7b846fba1f05fa8fa5ae6c5" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43385 } }
I20260811 11:59:23.321841 26542 raft_consensus.cc:348] T 73dcf39593364c67ac6f1670a34ff018 P 24cf6f1f898c42f48bd004c5591a2662 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51284b98c44a41f9b321cf3e93a9f0ad" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 35581 } } peers { permanent_uuid: "24cf6f1f898c42f48bd004c5591a2662" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 33427 } } peers { permanent_uuid: "87e4ddbef7b846fba1f05fa8fa5ae6c5" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43385 } }
I20260811 11:59:23.322085 26542 raft_consensus.cc:374] T 73dcf39593364c67ac6f1670a34ff018 P 24cf6f1f898c42f48bd004c5591a2662 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:23.322153 26542 raft_consensus.cc:729] T 73dcf39593364c67ac6f1670a34ff018 P 24cf6f1f898c42f48bd004c5591a2662 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 24cf6f1f898c42f48bd004c5591a2662, State: Initialized, Role: FOLLOWER
I20260811 11:59:23.322317 26543 ts_tablet_manager.cc:1428] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260811 11:59:23.322580 26542 consensus_queue.cc:260] T 73dcf39593364c67ac6f1670a34ff018 P 24cf6f1f898c42f48bd004c5591a2662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51284b98c44a41f9b321cf3e93a9f0ad" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 35581 } } peers { permanent_uuid: "24cf6f1f898c42f48bd004c5591a2662" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 33427 } } peers { permanent_uuid: "87e4ddbef7b846fba1f05fa8fa5ae6c5" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43385 } }
I20260811 11:59:23.323195 26239 heartbeater.cc:503] Master 127.6.143.190:37107 was elected leader, sending a full tablet report...
I20260811 11:59:23.323313 26382 heartbeater.cc:503] Master 127.6.143.190:37107 was elected leader, sending a full tablet report...
I20260811 11:59:23.324432 26542 ts_tablet_manager.cc:1428] T 73dcf39593364c67ac6f1670a34ff018 P 24cf6f1f898c42f48bd004c5591a2662: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
W20260811 11:59:23.369688 26240 tablet.cc:2369] T 73dcf39593364c67ac6f1670a34ff018 P 24cf6f1f898c42f48bd004c5591a2662: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 11:59:23.500813 26525 tablet.cc:2369] T 73dcf39593364c67ac6f1670a34ff018 P 51284b98c44a41f9b321cf3e93a9f0ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:59:23.510475 26548 raft_consensus.cc:482] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:59:23.510667 26548 raft_consensus.cc:504] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51284b98c44a41f9b321cf3e93a9f0ad" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 35581 } } peers { permanent_uuid: "24cf6f1f898c42f48bd004c5591a2662" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 33427 } } peers { permanent_uuid: "87e4ddbef7b846fba1f05fa8fa5ae6c5" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43385 } }
I20260811 11:59:23.511832 26548 leader_election.cc:288] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 51284b98c44a41f9b321cf3e93a9f0ad (127.6.143.131:35581), 24cf6f1f898c42f48bd004c5591a2662 (127.6.143.129:33427)
I20260811 11:59:23.515622 26547 raft_consensus.cc:482] T 73dcf39593364c67ac6f1670a34ff018 P 51284b98c44a41f9b321cf3e93a9f0ad [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:59:23.515805 26547 raft_consensus.cc:504] T 73dcf39593364c67ac6f1670a34ff018 P 51284b98c44a41f9b321cf3e93a9f0ad [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51284b98c44a41f9b321cf3e93a9f0ad" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 35581 } } peers { permanent_uuid: "24cf6f1f898c42f48bd004c5591a2662" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 33427 } } peers { permanent_uuid: "87e4ddbef7b846fba1f05fa8fa5ae6c5" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43385 } }
I20260811 11:59:23.515947 26385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:23.512107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:35581 (local address 127.6.143.130:40615)
0811 11:59:23.512267 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:23.512284 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:23.512401 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:23.512714 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:23.512716 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:23.512735 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:23.512908 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:23.512916 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.513808 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.513811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:23.514668 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:23.514678 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.515455 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.515459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:23.515562 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:23.515589 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 11:59:23.515672 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 11:59:23.515775 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":70,"spinlock_wait_cycles":12544}
I20260811 11:59:23.515975 26538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:23.512155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:40615 (local address 127.6.143.131:35581)
0811 11:59:23.512286 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:23.512293 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:23.512370 (+ 77us) server_negotiation.cc:407] Connection header received
0811 11:59:23.512538 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:23.512541 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:23.512594 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:23.512682 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:23.513063 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.513671 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.514802 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.515312 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.515469 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:23.515515 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:23.515701 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 11:59:23.515769 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 11:59:23.516332 26539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:23.512237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:33831 (local address 127.6.143.129:33427)
0811 11:59:23.512357 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:23.512363 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:23.512776 (+ 413us) server_negotiation.cc:407] Connection header received
0811 11:59:23.512915 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:23.512918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:23.512978 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:23.513079 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:23.513358 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.513942 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.515105 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.515592 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.515704 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:23.515745 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:23.516154 (+ 409us) server_negotiation.cc:299] Negotiation successful
0811 11:59:23.516225 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 11:59:23.516628 26476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73dcf39593364c67ac6f1670a34ff018" candidate_uuid: "87e4ddbef7b846fba1f05fa8fa5ae6c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51284b98c44a41f9b321cf3e93a9f0ad" is_pre_election: true
I20260811 11:59:23.516726 26550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:23.512119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:33427 (local address 127.6.143.130:33831)
0811 11:59:23.512681 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:23.512694 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:23.512802 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:23.513097 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:23.513100 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:23.513118 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:23.513239 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:23.513245 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.514052 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.514055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:23.514977 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:23.514988 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.515882 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.515886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:23.516002 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:23.516026 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 11:59:23.516519 (+ 493us) client_negotiation.cc:240] Negotiation successful
0811 11:59:23.516591 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"mutex_wait_us":144,"thread_start_us":156,"threads_started":1}
I20260811 11:59:23.516877 26476 raft_consensus.cc:2454] T 73dcf39593364c67ac6f1670a34ff018 P 51284b98c44a41f9b321cf3e93a9f0ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87e4ddbef7b846fba1f05fa8fa5ae6c5 in term 0.
I20260811 11:59:23.517055 26547 leader_election.cc:288] T 73dcf39593364c67ac6f1670a34ff018 P 51284b98c44a41f9b321cf3e93a9f0ad [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 24cf6f1f898c42f48bd004c5591a2662 (127.6.143.129:33427), 87e4ddbef7b846fba1f05fa8fa5ae6c5 (127.6.143.130:43385)
I20260811 11:59:23.517351 26261 leader_election.cc:302] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51284b98c44a41f9b321cf3e93a9f0ad, 87e4ddbef7b846fba1f05fa8fa5ae6c5; no voters:
I20260811 11:59:23.517344 26190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73dcf39593364c67ac6f1670a34ff018" candidate_uuid: "87e4ddbef7b846fba1f05fa8fa5ae6c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24cf6f1f898c42f48bd004c5591a2662" is_pre_election: true
I20260811 11:59:23.517639 26190 raft_consensus.cc:2454] T 73dcf39593364c67ac6f1670a34ff018 P 24cf6f1f898c42f48bd004c5591a2662 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87e4ddbef7b846fba1f05fa8fa5ae6c5 in term 0.
I20260811 11:59:23.517660 26548 raft_consensus.cc:2781] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:59:23.517738 26548 raft_consensus.cc:482] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:59:23.517786 26548 raft_consensus.cc:3037] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:23.519140 26548 raft_consensus.cc:504] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51284b98c44a41f9b321cf3e93a9f0ad" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 35581 } } peers { permanent_uuid: "24cf6f1f898c42f48bd004c5591a2662" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 33427 } } peers { permanent_uuid: "87e4ddbef7b846fba1f05fa8fa5ae6c5" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43385 } }
I20260811 11:59:23.519804 26190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73dcf39593364c67ac6f1670a34ff018" candidate_uuid: "87e4ddbef7b846fba1f05fa8fa5ae6c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24cf6f1f898c42f48bd004c5591a2662"
I20260811 11:59:23.519824 26476 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73dcf39593364c67ac6f1670a34ff018" candidate_uuid: "87e4ddbef7b846fba1f05fa8fa5ae6c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "51284b98c44a41f9b321cf3e93a9f0ad"
I20260811 11:59:23.519975 26476 raft_consensus.cc:3037] T 73dcf39593364c67ac6f1670a34ff018 P 51284b98c44a41f9b321cf3e93a9f0ad [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:23.520035 26190 raft_consensus.cc:3037] T 73dcf39593364c67ac6f1670a34ff018 P 24cf6f1f898c42f48bd004c5591a2662 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:23.521416 26190 raft_consensus.cc:2454] T 73dcf39593364c67ac6f1670a34ff018 P 24cf6f1f898c42f48bd004c5591a2662 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87e4ddbef7b846fba1f05fa8fa5ae6c5 in term 1.
I20260811 11:59:23.521437 26476 raft_consensus.cc:2454] T 73dcf39593364c67ac6f1670a34ff018 P 51284b98c44a41f9b321cf3e93a9f0ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 87e4ddbef7b846fba1f05fa8fa5ae6c5 in term 1.
I20260811 11:59:23.521761 26258 leader_election.cc:302] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 24cf6f1f898c42f48bd004c5591a2662, 87e4ddbef7b846fba1f05fa8fa5ae6c5; no voters:
I20260811 11:59:23.522162 26551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:23.517320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:43385 (local address 127.6.143.131:41445)
0811 11:59:23.517901 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:23.517913 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:23.518011 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:23.518348 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:23.518351 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:23.518367 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:23.518498 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:23.518512 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.519217 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.519221 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:23.520445 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:23.520455 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.521690 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.521693 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:23.521800 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:23.521826 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 11:59:23.521952 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 11:59:23.522028 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":4,"thread_start_us":122,"threads_started":1}
I20260811 11:59:23.522230 26540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:23.517647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:41445 (local address 127.6.143.130:43385)
0811 11:59:23.517761 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:23.517766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:23.518009 (+ 243us) server_negotiation.cc:407] Connection header received
0811 11:59:23.518138 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:23.518141 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:23.518184 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:23.518229 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:23.518627 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.519094 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.520587 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.521149 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.521258 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:23.521338 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:23.522011 (+ 673us) server_negotiation.cc:299] Negotiation successful
0811 11:59:23.522090 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 11:59:23.522497 26526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:23.517181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:33427 (local address 127.6.143.131:57509)
0811 11:59:23.517338 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:23.517354 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:23.517425 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:23.518086 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:23.518089 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:23.518110 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:23.518210 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:23.518217 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.519101 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.519104 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:23.520506 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:23.520516 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.522006 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.522019 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:23.522139 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:23.522165 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 11:59:23.522292 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 11:59:23.522375 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":85}
I20260811 11:59:23.522637 26552 raft_consensus.cc:2781] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:23.522742 26333 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73dcf39593364c67ac6f1670a34ff018" candidate_uuid: "51284b98c44a41f9b321cf3e93a9f0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87e4ddbef7b846fba1f05fa8fa5ae6c5" is_pre_election: true
I20260811 11:59:23.522831 26548 leader_election.cc:288] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5 [CANDIDATE]: Term 1 election: Requested vote from peers 51284b98c44a41f9b321cf3e93a9f0ad (127.6.143.131:35581), 24cf6f1f898c42f48bd004c5591a2662 (127.6.143.129:33427)
I20260811 11:59:23.522895 26552 raft_consensus.cc:686] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5 [term 1 LEADER]: Becoming Leader. State: Replica: 87e4ddbef7b846fba1f05fa8fa5ae6c5, State: Running, Role: LEADER
I20260811 11:59:23.523393 26539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:23.517610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:57509 (local address 127.6.143.129:33427)
0811 11:59:23.517828 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:23.517833 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:23.517846 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:59:23.517888 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:23.517890 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:23.517938 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:23.518004 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:23.518337 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.518977 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.520643 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.521510 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.521651 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:23.521723 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:23.523232 (+ 1509us) server_negotiation.cc:299] Negotiation successful
0811 11:59:23.523314 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260811 11:59:23.523344 26552 consensus_queue.cc:237] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51284b98c44a41f9b321cf3e93a9f0ad" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 35581 } } peers { permanent_uuid: "24cf6f1f898c42f48bd004c5591a2662" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 33427 } } peers { permanent_uuid: "87e4ddbef7b846fba1f05fa8fa5ae6c5" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43385 } }
I20260811 11:59:23.523838 26190 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73dcf39593364c67ac6f1670a34ff018" candidate_uuid: "51284b98c44a41f9b321cf3e93a9f0ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "24cf6f1f898c42f48bd004c5591a2662" is_pre_election: true
I20260811 11:59:23.524065 26190 raft_consensus.cc:2379] T 73dcf39593364c67ac6f1670a34ff018 P 24cf6f1f898c42f48bd004c5591a2662 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 51284b98c44a41f9b321cf3e93a9f0ad in current term 1: Already voted for candidate 87e4ddbef7b846fba1f05fa8fa5ae6c5 in this term.
I20260811 11:59:23.526793 26402 leader_election.cc:302] T 73dcf39593364c67ac6f1670a34ff018 P 51284b98c44a41f9b321cf3e93a9f0ad [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 51284b98c44a41f9b321cf3e93a9f0ad; no voters: 24cf6f1f898c42f48bd004c5591a2662, 87e4ddbef7b846fba1f05fa8fa5ae6c5
I20260811 11:59:23.527112 26547 raft_consensus.cc:2726] T 73dcf39593364c67ac6f1670a34ff018 P 51284b98c44a41f9b321cf3e93a9f0ad [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 11:59:23.527424 26020 catalog_manager.cc:5495] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 87e4ddbef7b846fba1f05fa8fa5ae6c5 (127.6.143.130). New cstate: current_term: 1 leader_uuid: "87e4ddbef7b846fba1f05fa8fa5ae6c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51284b98c44a41f9b321cf3e93a9f0ad" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 35581 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "24cf6f1f898c42f48bd004c5591a2662" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 33427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87e4ddbef7b846fba1f05fa8fa5ae6c5" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 43385 } health_report { overall_health: HEALTHY } } }
W20260811 11:59:23.564479 26384 tablet.cc:2369] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:59:23.588090 26540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:23.584730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59662 (local address 127.6.143.130:43385)
0811 11:59:23.584867 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:23.584871 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:23.584884 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:59:23.584991 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:23.584993 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:23.585049 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:23.585143 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:59:23.585397 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.585885 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.586735 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:23.586949 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:23.587024 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:23.587299 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:59:23.587730 (+ 431us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:59:23.587822 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:23.587923 (+ 101us) server_negotiation.cc:299] Negotiation successful
0811 11:59:23.587968 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 11:59:23.594447 26190 raft_consensus.cc:1261] T 73dcf39593364c67ac6f1670a34ff018 P 24cf6f1f898c42f48bd004c5591a2662 [term 1 FOLLOWER]: Refusing update from remote peer 87e4ddbef7b846fba1f05fa8fa5ae6c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:59:23.594485 26476 raft_consensus.cc:1261] T 73dcf39593364c67ac6f1670a34ff018 P 51284b98c44a41f9b321cf3e93a9f0ad [term 1 FOLLOWER]: Refusing update from remote peer 87e4ddbef7b846fba1f05fa8fa5ae6c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:59:23.595224 26552 consensus_queue.cc:1035] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "51284b98c44a41f9b321cf3e93a9f0ad" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 35581 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:59:23.595454 26553 consensus_queue.cc:1035] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "24cf6f1f898c42f48bd004c5591a2662" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 33427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:59:23.606060 26559 mvcc.cc:205] Tried to move back new op lower bound from 7317297412473741312 to 7317297412194865152. Current Snapshot: MvccSnapshot[applied={T|T < 7317297412473741312}]
I20260811 11:59:23.614219 26020 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:59:23.614400 26020 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35968:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:59:23.617003 26020 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 059517940edd436ea975b86d34bce457: Sending DeleteTablet for 3 replicas of tablet 73dcf39593364c67ac6f1670a34ff018
I20260811 11:59:23.617812 26166 tablet_service.cc:1514] Processing DeleteTablet for tablet 73dcf39593364c67ac6f1670a34ff018 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:59:23 UTC) from {username='slave'} at 127.0.0.1:51822
I20260811 11:59:23.617842 26453 tablet_service.cc:1514] Processing DeleteTablet for tablet 73dcf39593364c67ac6f1670a34ff018 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:59:23 UTC) from {username='slave'} at 127.0.0.1:34270
I20260811 11:59:23.618199 26312 tablet_service.cc:1514] Processing DeleteTablet for tablet 73dcf39593364c67ac6f1670a34ff018 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:59:23 UTC) from {username='slave'} at 127.0.0.1:59646
I20260811 11:59:23.618709 26570 tablet_replica.cc:330] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5: stopping tablet replica
I20260811 11:59:23.619033 26570 raft_consensus.cc:2229] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5 [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:59:23.619441 26570 raft_consensus.cc:2258] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:59:23.619470 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 26090
I20260811 11:59:23.624058 26569 tablet_replica.cc:330] T 73dcf39593364c67ac6f1670a34ff018 P 51284b98c44a41f9b321cf3e93a9f0ad: stopping tablet replica
I20260811 11:59:23.624454 26570 ts_tablet_manager.cc:1905] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:59:23.624461 26569 raft_consensus.cc:2229] T 73dcf39593364c67ac6f1670a34ff018 P 51284b98c44a41f9b321cf3e93a9f0ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:59:23.624787 26569 raft_consensus.cc:2258] T 73dcf39593364c67ac6f1670a34ff018 P 51284b98c44a41f9b321cf3e93a9f0ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:59:23.626343 26569 ts_tablet_manager.cc:1905] T 73dcf39593364c67ac6f1670a34ff018 P 51284b98c44a41f9b321cf3e93a9f0ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:59:23.628280 26570 ts_tablet_manager.cc:1918] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:59:23.628393 26570 log.cc:1196] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-1/wal/wals/73dcf39593364c67ac6f1670a34ff018
I20260811 11:59:23.628762 26570 ts_tablet_manager.cc:1939] T 73dcf39593364c67ac6f1670a34ff018 P 87e4ddbef7b846fba1f05fa8fa5ae6c5: Deleting consensus metadata
I20260811 11:59:23.629165 26569 ts_tablet_manager.cc:1918] T 73dcf39593364c67ac6f1670a34ff018 P 51284b98c44a41f9b321cf3e93a9f0ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:59:23.629264 26569 log.cc:1196] T 73dcf39593364c67ac6f1670a34ff018 P 51284b98c44a41f9b321cf3e93a9f0ad: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786449319379472-6718-0/minicluster-data/ts-2/wal/wals/73dcf39593364c67ac6f1670a34ff018
I20260811 11:59:23.629583 26569 ts_tablet_manager.cc:1939] T 73dcf39593364c67ac6f1670a34ff018 P 51284b98c44a41f9b321cf3e93a9f0ad: Deleting consensus metadata
I20260811 11:59:23.630045 26001 catalog_manager.cc:4841] TS 87e4ddbef7b846fba1f05fa8fa5ae6c5 (127.6.143.130:43385): tablet 73dcf39593364c67ac6f1670a34ff018 (table test-table [id=062c37e7bb9945738f44bb34ab5d7f3e]) successfully deleted
I20260811 11:59:23.630818 26005 catalog_manager.cc:4841] TS 51284b98c44a41f9b321cf3e93a9f0ad (127.6.143.131:35581): tablet 73dcf39593364c67ac6f1670a34ff018 (table test-table [id=062c37e7bb9945738f44bb34ab5d7f3e]) successfully deleted
W20260811 11:59:23.633671 26000 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.143.129:33427 (error 108)
I20260811 11:59:23.633946 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 26242
I20260811 11:59:23.637086 26536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:23.636632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:33427 (local address 127.0.0.1:51832)
0811 11:59:23.636829 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:23.636942 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.129:33427: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260811 11:59:23.637272 26000 catalog_manager.cc:4584] TS 24cf6f1f898c42f48bd004c5591a2662 (127.6.143.129:33427): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 73dcf39593364c67ac6f1670a34ff018: Network error: Client connection negotiation failed: client connection to 127.6.143.129:33427: connect: Connection refused (error 111)
I20260811 11:59:23.642737 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 26386
I20260811 11:59:23.651083 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 25983
2026-08-11T11:59:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1254 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:59:23 dist-test-slave-zkpd krb5kdc[26574](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:59:23 dist-test-slave-zkpd krb5kdc[26574](info): set up 2 sockets
Aug 11 11:59:23 dist-test-slave-zkpd krb5kdc[26574](info): commencing operation
krb5kdc: starting...
W20260811 11:59:24.737484 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:59:24 dist-test-slave-zkpd krb5kdc[26574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:59:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:59:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
I20260811 11:59:24.866218 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:35657
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:43931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:35657
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:24.985478 26587 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:24.985720 26587 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:24.985759 26587 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:24.990248 26587 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:59:24.990301 26587 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:24.990321 26587 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:24.990337 26587 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:59:24.990355 26587 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:59:24.995407 26587 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:43931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:35657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:35657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26587
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:24.996421 26587 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:24.997345 26587 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:25.006018 26595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:25.006085 26587 server_base.cc:1034] running on GCE node
W20260811 11:59:25.006029 26597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:25.006039 26594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:25.006853 26587 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:25.007862 26587 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:25.009104 26587 hybrid_clock.cc:648] HybridClock initialized: now 1786449565009087 us; error 25 us; skew 500 ppm
Aug 11 11:59:25 dist-test-slave-zkpd krb5kdc[26574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449565, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:25.016433 26587 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:59:25.017367 26587 webserver.cc:460] Webserver started at http://127.6.143.190:36259/ using document root <none> and password file <none>
I20260811 11:59:25.017948 26587 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:25.018044 26587 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:25.018278 26587 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:25.020349 26587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "f612d51471d9435a85900932a2158369"
format_stamp: "Formatted at 2026-08-11 11:59:25 on dist-test-slave-zkpd"
server_key: "da8cc3e17039bbca011c9637e669e997"
server_key_iv: "5262221ed151d57681eeb6b269220db2"
server_key_version: "encryptionkey@0"
I20260811 11:59:25.020924 26587 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "f612d51471d9435a85900932a2158369"
format_stamp: "Formatted at 2026-08-11 11:59:25 on dist-test-slave-zkpd"
server_key: "da8cc3e17039bbca011c9637e669e997"
server_key_iv: "5262221ed151d57681eeb6b269220db2"
server_key_version: "encryptionkey@0"
I20260811 11:59:25.024694 26587 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 11:59:25.027145 26604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:25.028523 26587 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:59:25.028672 26587 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "f612d51471d9435a85900932a2158369"
format_stamp: "Formatted at 2026-08-11 11:59:25 on dist-test-slave-zkpd"
server_key: "da8cc3e17039bbca011c9637e669e997"
server_key_iv: "5262221ed151d57681eeb6b269220db2"
server_key_version: "encryptionkey@0"
I20260811 11:59:25.028788 26587 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:25.046116 26587 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:25.048992 26587 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:25.049213 26587 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:25.056856 26587 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:35657
I20260811 11:59:25.056890 26661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:35657 every 8 connection(s)
I20260811 11:59:25.057866 26587 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:59:25.060716 26663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:25.062820 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 26587
I20260811 11:59:25.062971 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:59:25.063328 6718 external_mini_cluster.cc:1428] Setting key f0a6e9cb5a1391e02b36bc1dcc43c3bd
I20260811 11:59:25.067351 26663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f612d51471d9435a85900932a2158369: Bootstrap starting.
I20260811 11:59:25.069597 26663 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f612d51471d9435a85900932a2158369: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:25.070405 26663 log.cc:824] T 00000000000000000000000000000000 P f612d51471d9435a85900932a2158369: Log is configured to *not* fsync() on all Append() calls
Aug 11 11:59:25 dist-test-slave-zkpd krb5kdc[26574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:59:25.072388 26663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f612d51471d9435a85900932a2158369: No bootstrap required, opened a new log
I20260811 11:59:25.076000 26663 raft_consensus.cc:348] T 00000000000000000000000000000000 P f612d51471d9435a85900932a2158369 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f612d51471d9435a85900932a2158369" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 35657 } }
I20260811 11:59:25.076264 26663 raft_consensus.cc:374] T 00000000000000000000000000000000 P f612d51471d9435a85900932a2158369 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:25.076345 26663 raft_consensus.cc:729] T 00000000000000000000000000000000 P f612d51471d9435a85900932a2158369 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f612d51471d9435a85900932a2158369, State: Initialized, Role: FOLLOWER
I20260811 11:59:25.076779 26663 consensus_queue.cc:260] T 00000000000000000000000000000000 P f612d51471d9435a85900932a2158369 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f612d51471d9435a85900932a2158369" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 35657 } }
I20260811 11:59:25.076934 26663 raft_consensus.cc:388] T 00000000000000000000000000000000 P f612d51471d9435a85900932a2158369 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:59:25.077023 26663 raft_consensus.cc:482] T 00000000000000000000000000000000 P f612d51471d9435a85900932a2158369 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:59:25.077147 26663 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f612d51471d9435a85900932a2158369 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:25.077318 26666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:25.064762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47868 (local address 127.6.143.190:35657)
0811 11:59:25.065185 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:25.065200 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:25.065234 (+ 34us) server_negotiation.cc:407] Connection header received
0811 11:59:25.065863 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:25.065879 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:25.066190 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:25.066532 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:25.067372 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:25.068458 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:25.069310 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:25.069840 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:25.070046 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:25.072563 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:25.072568 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:25.072580 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:25.072605 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:25.074242 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:25.074747 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:25.074752 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:25.074757 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:25.074864 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:25.075176 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:25.075180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:25.075182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:25.075541 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:25.075752 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:25.076043 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 11:59:25.076391 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":130,"threads_started":1}
I20260811 11:59:25.078045 26663 raft_consensus.cc:504] T 00000000000000000000000000000000 P f612d51471d9435a85900932a2158369 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f612d51471d9435a85900932a2158369" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 35657 } }
I20260811 11:59:25.078413 26663 leader_election.cc:302] T 00000000000000000000000000000000 P f612d51471d9435a85900932a2158369 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f612d51471d9435a85900932a2158369; no voters:
I20260811 11:59:25.078740 26663 leader_election.cc:288] T 00000000000000000000000000000000 P f612d51471d9435a85900932a2158369 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:59:25.078850 26668 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f612d51471d9435a85900932a2158369 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:25.079067 26668 raft_consensus.cc:686] T 00000000000000000000000000000000 P f612d51471d9435a85900932a2158369 [term 1 LEADER]: Becoming Leader. State: Replica: f612d51471d9435a85900932a2158369, State: Running, Role: LEADER
I20260811 11:59:25.079365 26668 consensus_queue.cc:237] T 00000000000000000000000000000000 P f612d51471d9435a85900932a2158369 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f612d51471d9435a85900932a2158369" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 35657 } }
I20260811 11:59:25.079984 26663 sys_catalog.cc:564] T 00000000000000000000000000000000 P f612d51471d9435a85900932a2158369 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:59:25.080587 26670 sys_catalog.cc:455] T 00000000000000000000000000000000 P f612d51471d9435a85900932a2158369 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f612d51471d9435a85900932a2158369. Latest consensus state: current_term: 1 leader_uuid: "f612d51471d9435a85900932a2158369" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f612d51471d9435a85900932a2158369" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 35657 } } }
I20260811 11:59:25.080745 26670 sys_catalog.cc:458] T 00000000000000000000000000000000 P f612d51471d9435a85900932a2158369 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:25.081063 26669 sys_catalog.cc:455] T 00000000000000000000000000000000 P f612d51471d9435a85900932a2158369 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f612d51471d9435a85900932a2158369" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f612d51471d9435a85900932a2158369" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 35657 } } }
I20260811 11:59:25.081168 26669 sys_catalog.cc:458] T 00000000000000000000000000000000 P f612d51471d9435a85900932a2158369 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:25.081588 26675 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:59:25.084785 26675 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:59:25.089813 26675 catalog_manager.cc:1347] Generated new cluster ID: ad5d46fab7c24449a9f96e3f2507711f
I20260811 11:59:25.089874 26675 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:59:25.129360 26675 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:59:25.130507 26675 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:59:25.135571 26675 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f612d51471d9435a85900932a2158369: Generated new TSK 0
I20260811 11:59:25.136307 26675 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
I20260811 11:59:25.169188 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:0
--local_ip_for_outbound_sockets=127.6.143.129
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:35657
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:43931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:25.287470 26690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:25.287709 26690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:25.287751 26690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:25.291927 26690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:25.291987 26690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:25.292044 26690 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:59:25.297411 26690 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:43931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:35657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:25.298354 26690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:25.299304 26690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:25.309068 26699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:25.309077 26697 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:25.309329 26701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:25.309343 26690 server_base.cc:1034] running on GCE node
I20260811 11:59:25.309798 26690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:25.310395 26690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:25.311558 26690 hybrid_clock.cc:648] HybridClock initialized: now 1786449565311542 us; error 24 us; skew 500 ppm
Aug 11 11:59:25 dist-test-slave-zkpd krb5kdc[26574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449565, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:25.314455 26690 init.cc:377] Logged in from keytab as kudu/127.6.143.129@KRBTEST.COM (short username kudu)
I20260811 11:59:25.315248 26690 webserver.cc:460] Webserver started at http://127.6.143.129:41543/ using document root <none> and password file <none>
I20260811 11:59:25.315794 26690 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:25.315887 26690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:25.316159 26690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:25.318115 26690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-0/data/instance:
uuid: "af05ba16c4d24be893f3a4d151749ecb"
format_stamp: "Formatted at 2026-08-11 11:59:25 on dist-test-slave-zkpd"
server_key: "ab80bbf61d639173505c9d781c82dbb0"
server_key_iv: "c7a9a3479f0a4c4fee74efdb147aebf9"
server_key_version: "encryptionkey@0"
I20260811 11:59:25.318673 26690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance:
uuid: "af05ba16c4d24be893f3a4d151749ecb"
format_stamp: "Formatted at 2026-08-11 11:59:25 on dist-test-slave-zkpd"
server_key: "ab80bbf61d639173505c9d781c82dbb0"
server_key_iv: "c7a9a3479f0a4c4fee74efdb147aebf9"
server_key_version: "encryptionkey@0"
I20260811 11:59:25.322515 26690 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 11:59:25.325083 26710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:25.326301 26690 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 11:59:25.326448 26690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "af05ba16c4d24be893f3a4d151749ecb"
format_stamp: "Formatted at 2026-08-11 11:59:25 on dist-test-slave-zkpd"
server_key: "ab80bbf61d639173505c9d781c82dbb0"
server_key_iv: "c7a9a3479f0a4c4fee74efdb147aebf9"
server_key_version: "encryptionkey@0"
I20260811 11:59:25.326563 26690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:25.349093 26690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:25.351938 26690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:25.352138 26690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:25.352720 26690 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:59:25.353547 26690 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:59:25.353626 26690 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:25.353711 26690 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:59:25.353765 26690 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:25.362457 26690 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:36603
I20260811 11:59:25.362483 26832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:36603 every 8 connection(s)
I20260811 11:59:25.363332 26690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:59:25.366652 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 26690
I20260811 11:59:25.366806 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance
I20260811 11:59:25.367110 6718 external_mini_cluster.cc:1428] Setting key 81aa91dc3749bb597a76b75236a8f19a
Aug 11 11:59:25 dist-test-slave-zkpd krb5kdc[26574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449565, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.143.130@KRBTEST.COM; defaulting to no policy
I20260811 11:59:25.380746 26666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:25.365203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:41153 (local address 127.6.143.190:35657)
0811 11:59:25.365363 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:25.365367 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:25.366159 (+ 792us) server_negotiation.cc:407] Connection header received
0811 11:59:25.367589 (+ 1430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:25.367592 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:25.367661 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:25.367757 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:25.369407 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:25.370213 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:25.372073 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:25.372480 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:25.372603 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:25.376361 (+ 3758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:25.376368 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:25.376371 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:25.376406 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:25.378140 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:25.378810 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:25.378812 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:25.378813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:25.378887 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:25.379303 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:25.379305 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:25.379306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:25.379470 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:25.379543 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:25.380341 (+ 798us) server_negotiation.cc:299] Negotiation successful
0811 11:59:25.380505 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 11:59:25.381910 26835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:25.365555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:35657 (local address 127.6.143.129:41153)
0811 11:59:25.366003 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:25.366046 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:25.367351 (+ 1305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:25.367997 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:25.368006 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:25.368534 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:59:25.369231 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:25.369255 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:25.370357 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:25.370360 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:25.371938 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:25.371954 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:25.372793 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:25.372800 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:25.373029 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:25.373779 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:59:25.373802 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:59:25.376123 (+ 2321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:59:25.378267 (+ 2144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:25.378276 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:25.378296 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:25.378675 (+ 379us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:25.378986 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:25.378991 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:25.378993 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:25.379194 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:25.379555 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:59:25.379563 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:59:25.380005 (+ 442us) client_negotiation.cc:769] Sending connection context
0811 11:59:25.380229 (+ 224us) client_negotiation.cc:240] Negotiation successful
0811 11:59:25.380710 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":106,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.130@KRBTEST.COM" created.
I20260811 11:59:25.383325 26833 heartbeater.cc:347] Connected to a master server at 127.6.143.190:35657
I20260811 11:59:25.383580 26833 heartbeater.cc:464] Registering TS with master...
I20260811 11:59:25.384200 26833 heartbeater.cc:511] Master 127.6.143.190:35657 requested a full tablet report, sending...
I20260811 11:59:25.385823 26624 ts_manager.cc:194] Registered new tserver with Master: af05ba16c4d24be893f3a4d151749ecb (127.6.143.129:36603)
I20260811 11:59:25.386974 26624 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.129@KRBTEST.COM'} at 127.6.143.129:41153
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
I20260811 11:59:25.398205 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:0
--local_ip_for_outbound_sockets=127.6.143.130
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:35657
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:43931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:25.514216 26838 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:25.514436 26838 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:25.514477 26838 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:25.518299 26838 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:25.518344 26838 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:25.518399 26838 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:59:25.523276 26838 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:43931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:35657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26838
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:25.524250 26838 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:25.525197 26838 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:25.534716 26846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:25.534716 26848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:25.534755 26845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:25.535048 26838 server_base.cc:1034] running on GCE node
I20260811 11:59:25.535590 26838 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:25.536338 26838 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:25.537530 26838 hybrid_clock.cc:648] HybridClock initialized: now 1786449565537507 us; error 30 us; skew 500 ppm
Aug 11 11:59:25 dist-test-slave-zkpd krb5kdc[26574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449565, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:25.540331 26838 init.cc:377] Logged in from keytab as kudu/127.6.143.130@KRBTEST.COM (short username kudu)
I20260811 11:59:25.541141 26838 webserver.cc:460] Webserver started at http://127.6.143.130:33101/ using document root <none> and password file <none>
I20260811 11:59:25.541676 26838 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:25.541733 26838 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:25.541921 26838 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:25.543977 26838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-1/data/instance:
uuid: "2158f634215c484e9f5eb7c1fb8c4fbd"
format_stamp: "Formatted at 2026-08-11 11:59:25 on dist-test-slave-zkpd"
server_key: "7066af605621f93900fdd190e3b9262a"
server_key_iv: "faa838257925e74d441e26551ab9b96e"
server_key_version: "encryptionkey@0"
I20260811 11:59:25.544566 26838 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance:
uuid: "2158f634215c484e9f5eb7c1fb8c4fbd"
format_stamp: "Formatted at 2026-08-11 11:59:25 on dist-test-slave-zkpd"
server_key: "7066af605621f93900fdd190e3b9262a"
server_key_iv: "faa838257925e74d441e26551ab9b96e"
server_key_version: "encryptionkey@0"
I20260811 11:59:25.548147 26838 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 11:59:25.550498 26855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:25.551689 26838 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:59:25.551823 26838 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "2158f634215c484e9f5eb7c1fb8c4fbd"
format_stamp: "Formatted at 2026-08-11 11:59:25 on dist-test-slave-zkpd"
server_key: "7066af605621f93900fdd190e3b9262a"
server_key_iv: "faa838257925e74d441e26551ab9b96e"
server_key_version: "encryptionkey@0"
I20260811 11:59:25.551978 26838 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:25.562357 26838 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:25.565147 26838 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:25.565336 26838 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:25.565884 26838 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:59:25.566696 26838 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:59:25.566767 26838 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:25.566838 26838 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:59:25.566880 26838 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:25.575994 26838 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:42815
I20260811 11:59:25.576054 26976 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:42815 every 8 connection(s)
I20260811 11:59:25.577026 26838 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
I20260811 11:59:25.584975 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 26838
I20260811 11:59:25.585119 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance
I20260811 11:59:25.585502 6718 external_mini_cluster.cc:1428] Setting key 5a4c854a7c0bd3132ad7fbbac9930c00
Aug 11 11:59:25 dist-test-slave-zkpd krb5kdc[26574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449565, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:59:25.590972 26666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:25.578776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:50323 (local address 127.6.143.190:35657)
0811 11:59:25.578904 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:25.578908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:25.579720 (+ 812us) server_negotiation.cc:407] Connection header received
0811 11:59:25.581511 (+ 1791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:25.581514 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:25.581560 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:25.581632 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:25.582768 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:25.583285 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:25.584462 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:25.584668 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:25.584791 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:25.587483 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:25.587486 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:25.587488 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:25.587511 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:25.588956 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:25.589472 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:25.589476 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:25.589479 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:25.589562 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:25.589918 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:25.589922 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:25.589923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:25.590099 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:25.590207 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:25.590680 (+ 473us) server_negotiation.cc:299] Negotiation successful
0811 11:59:25.590796 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 11:59:25.591706 26979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:25.579142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:35657 (local address 127.6.143.130:50323)
0811 11:59:25.579571 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:25.579611 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:25.581331 (+ 1720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:25.581761 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:25.581767 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:25.582105 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:59:25.582632 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:25.582645 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:25.583389 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:25.583392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:25.584358 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:25.584368 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:25.584822 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:25.584825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:25.584960 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:25.585665 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:59:25.585686 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:59:25.587302 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:59:25.589080 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:25.589086 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:25.589100 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:25.589357 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:25.589657 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:25.589661 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:25.589663 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:25.589810 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:25.590207 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:59:25.590214 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:59:25.590448 (+ 234us) client_negotiation.cc:769] Sending connection context
0811 11:59:25.590633 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 11:59:25.590936 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":106,"threads_started":1}
I20260811 11:59:25.592878 26977 heartbeater.cc:347] Connected to a master server at 127.6.143.190:35657
I20260811 11:59:25.593129 26977 heartbeater.cc:464] Registering TS with master...
I20260811 11:59:25.593596 26977 heartbeater.cc:511] Master 127.6.143.190:35657 requested a full tablet report, sending...
I20260811 11:59:25.594638 26624 ts_manager.cc:194] Registered new tserver with Master: 2158f634215c484e9f5eb7c1fb8c4fbd (127.6.143.130:42815)
I20260811 11:59:25.595201 26624 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.130@KRBTEST.COM'} at 127.6.143.130:50323
WARNING: no policy specified for kudu/127.6.143.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
I20260811 11:59:25.614087 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:0
--local_ip_for_outbound_sockets=127.6.143.131
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:35657
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:43931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:25.727608 26982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:25.727880 26982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:25.727970 26982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:25.732209 26982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:25.732280 26982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:25.732363 26982 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:59:25.737457 26982 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:43931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:35657
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:25.738423 26982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:25.739395 26982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:25.748757 26988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:25.748745 26989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:25.748745 26991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:25.749127 26982 server_base.cc:1034] running on GCE node
I20260811 11:59:25.749503 26982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:25.750047 26982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:25.751204 26982 hybrid_clock.cc:648] HybridClock initialized: now 1786449565751189 us; error 24 us; skew 500 ppm
Aug 11 11:59:25 dist-test-slave-zkpd krb5kdc[26574](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449565, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:25.753890 26982 init.cc:377] Logged in from keytab as kudu/127.6.143.131@KRBTEST.COM (short username kudu)
I20260811 11:59:25.754675 26982 webserver.cc:460] Webserver started at http://127.6.143.131:42583/ using document root <none> and password file <none>
I20260811 11:59:25.755273 26982 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:25.755360 26982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:25.755591 26982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:25.757658 26982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-2/data/instance:
uuid: "1a52d40921fe4d9da6e7adc92299989c"
format_stamp: "Formatted at 2026-08-11 11:59:25 on dist-test-slave-zkpd"
server_key: "492317a67ef3a13e9ac0501fcb6898f5"
server_key_iv: "ad998db6fe77192f8701c4aaee44e96e"
server_key_version: "encryptionkey@0"
I20260811 11:59:25.758208 26982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance:
uuid: "1a52d40921fe4d9da6e7adc92299989c"
format_stamp: "Formatted at 2026-08-11 11:59:25 on dist-test-slave-zkpd"
server_key: "492317a67ef3a13e9ac0501fcb6898f5"
server_key_iv: "ad998db6fe77192f8701c4aaee44e96e"
server_key_version: "encryptionkey@0"
I20260811 11:59:25.761837 26982 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 11:59:25.764150 26998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:25.765306 26982 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:59:25.765439 26982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "1a52d40921fe4d9da6e7adc92299989c"
format_stamp: "Formatted at 2026-08-11 11:59:25 on dist-test-slave-zkpd"
server_key: "492317a67ef3a13e9ac0501fcb6898f5"
server_key_iv: "ad998db6fe77192f8701c4aaee44e96e"
server_key_version: "encryptionkey@0"
I20260811 11:59:25.765545 26982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:25.782991 26982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:25.785794 26982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:25.785997 26982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:25.786566 26982 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:59:25.787370 26982 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:59:25.787441 26982 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:25.787513 26982 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:59:25.787557 26982 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:25.796245 26982 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:43413
I20260811 11:59:25.796285 27117 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:43413 every 8 connection(s)
I20260811 11:59:25.797115 26982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:59:25.801054 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 26982
I20260811 11:59:25.801224 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance
I20260811 11:59:25.801590 6718 external_mini_cluster.cc:1428] Setting key 63093d8c54d98b14b0ea7a35e142b2df
Aug 11 11:59:25 dist-test-slave-zkpd krb5kdc[26574](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449565, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:59:25.811043 26666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:25.798943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:34149 (local address 127.6.143.190:35657)
0811 11:59:25.799083 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:25.799088 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:25.799910 (+ 822us) server_negotiation.cc:407] Connection header received
0811 11:59:25.801267 (+ 1357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:25.801272 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:25.801337 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:25.801456 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:25.803018 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:25.803598 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:25.804955 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:25.805168 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:25.805251 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:25.807705 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:25.807715 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:25.807718 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:25.807742 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:25.809060 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:25.809567 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:25.809572 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:25.809574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:25.809658 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:25.810006 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:25.810010 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:25.810012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:25.810188 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:25.810299 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:25.810768 (+ 469us) server_negotiation.cc:299] Negotiation successful
0811 11:59:25.810885 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 11:59:25.811810 27120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:25.799299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:35657 (local address 127.6.143.131:34149)
0811 11:59:25.799747 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:25.799788 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:25.801024 (+ 1236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:25.801646 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:25.801655 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:25.802212 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:59:25.802816 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:25.802858 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:25.803755 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:25.803758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:25.804835 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:25.804847 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:25.805303 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:25.805306 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:25.805450 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:25.806083 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:59:25.806101 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:59:25.807543 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:59:25.809182 (+ 1639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:25.809187 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:25.809202 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:25.809462 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:25.809754 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:25.809758 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:25.809760 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:25.809906 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:25.810298 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:59:25.810304 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:59:25.810538 (+ 234us) client_negotiation.cc:769] Sending connection context
0811 11:59:25.810726 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 11:59:25.811047 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":109,"threads_started":1}
I20260811 11:59:25.812932 27118 heartbeater.cc:347] Connected to a master server at 127.6.143.190:35657
I20260811 11:59:25.813182 27118 heartbeater.cc:464] Registering TS with master...
I20260811 11:59:25.813668 27118 heartbeater.cc:511] Master 127.6.143.190:35657 requested a full tablet report, sending...
I20260811 11:59:25.814721 26624 ts_manager.cc:194] Registered new tserver with Master: 1a52d40921fe4d9da6e7adc92299989c (127.6.143.131:43413)
I20260811 11:59:25.815338 26624 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.131@KRBTEST.COM'} at 127.6.143.131:34149
I20260811 11:59:25.815946 6718 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:59:25.825380 26666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:25.817945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47880 (local address 127.6.143.190:35657)
0811 11:59:25.818053 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:25.818059 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:25.818216 (+ 157us) server_negotiation.cc:407] Connection header received
0811 11:59:25.818309 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:25.818311 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:25.818344 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:25.818408 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:25.819045 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:25.819517 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:25.820359 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:25.820545 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:25.820621 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:25.822146 (+ 1525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:25.822149 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:25.822151 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:25.822167 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:25.823319 (+ 1152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:25.824056 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:25.824069 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:25.824071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:25.824163 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:25.824664 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:25.824667 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:25.824668 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:25.824850 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:25.824975 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:25.825204 (+ 229us) server_negotiation.cc:299] Negotiation successful
0811 11:59:25.825277 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 11:59:25.828505 26623 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47880:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:59:25.831741 26623 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:59:25.845923 27130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:25.839952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:42815 (local address 127.0.0.1:57144)
0811 11:59:25.840601 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:25.840670 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:25.840819 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:25.841440 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:25.841446 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:25.841469 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:25.841617 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:25.841626 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:25.843020 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:25.843024 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:25.844243 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:25.844255 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:25.845380 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:25.845384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:25.845502 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:25.845546 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 11:59:25.845651 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 11:59:25.845767 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"mutex_wait_us":28,"spinlock_wait_cycles":2176,"thread_start_us":87,"threads_started":1}
I20260811 11:59:25.846269 27135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:25.841103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58964 (local address 127.6.143.131:43413)
0811 11:59:25.841789 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:25.841803 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:25.841828 (+ 25us) server_negotiation.cc:407] Connection header received
0811 11:59:25.841903 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:25.841913 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:25.842046 (+ 133us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:25.842199 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:25.842409 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:25.843452 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:25.844893 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:25.845537 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:25.845656 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:25.845809 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:25.845870 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 11:59:25.845969 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":156,"threads_started":1}
I20260811 11:59:25.846385 27132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:25.840063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57144 (local address 127.6.143.130:42815)
0811 11:59:25.840427 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:25.840439 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:25.840971 (+ 532us) server_negotiation.cc:407] Connection header received
0811 11:59:25.841064 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:25.841070 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:25.841281 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:25.841436 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:25.841769 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:25.842887 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:25.844384 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:25.845239 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:25.845352 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:25.845521 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:25.846122 (+ 601us) server_negotiation.cc:299] Negotiation successful
0811 11:59:25.846233 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":180,"threads_started":1}
I20260811 11:59:25.846318 27131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:25.839983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:43413 (local address 127.0.0.1:58964)
0811 11:59:25.840650 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:25.840670 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:25.840819 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:25.842179 (+ 1360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:25.842182 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:25.842201 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:25.842299 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:25.842306 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:25.843595 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:25.843598 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:25.844761 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:25.844772 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:25.845669 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:25.845673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:25.845758 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:25.845783 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 11:59:25.845846 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 11:59:25.845919 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"mutex_wait_us":26,"spinlock_wait_cycles":2432,"thread_start_us":76,"threads_started":1}
I20260811 11:59:25.847332 27133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:25.840681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:36603 (local address 127.0.0.1:53828)
0811 11:59:25.841719 (+ 1038us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:25.841734 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:25.841829 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:25.842497 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:25.842500 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:25.842517 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:25.842617 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:25.842622 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:25.843880 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:25.843886 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:25.845346 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:25.845356 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:25.846909 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:25.846913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:25.847011 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:25.847048 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 11:59:25.847126 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 11:59:25.847208 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":956,"mutex_wait_us":1,"thread_start_us":97,"threads_started":1}
I20260811 11:59:25.847630 27134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:25.840882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53828 (local address 127.6.143.129:36603)
0811 11:59:25.841281 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:25.841299 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:25.841924 (+ 625us) server_negotiation.cc:407] Connection header received
0811 11:59:25.842009 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:25.842014 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:25.842210 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:25.842347 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:25.842749 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:25.843759 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:25.845493 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:25.846777 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:25.846887 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:25.847016 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:25.847176 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 11:59:25.847300 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":102,"threads_started":1}
I20260811 11:59:25.849552 26905 tablet_service.cc:1467] Processing CreateTablet for tablet fa94f6e70eeb4f41ab862270bf50b96c (DEFAULT_TABLE table=test-table [id=0013118e9619462c9011e3da64bc6bdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:59:25.849831 27047 tablet_service.cc:1467] Processing CreateTablet for tablet fa94f6e70eeb4f41ab862270bf50b96c (DEFAULT_TABLE table=test-table [id=0013118e9619462c9011e3da64bc6bdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:59:25.850028 26760 tablet_service.cc:1467] Processing CreateTablet for tablet fa94f6e70eeb4f41ab862270bf50b96c (DEFAULT_TABLE table=test-table [id=0013118e9619462c9011e3da64bc6bdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:59:25.850760 27047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa94f6e70eeb4f41ab862270bf50b96c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:25.850760 26905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa94f6e70eeb4f41ab862270bf50b96c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:25.851066 26760 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa94f6e70eeb4f41ab862270bf50b96c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:25.857630 27136 tablet_bootstrap.cc:492] T fa94f6e70eeb4f41ab862270bf50b96c P af05ba16c4d24be893f3a4d151749ecb: Bootstrap starting.
I20260811 11:59:25.858124 27137 tablet_bootstrap.cc:492] T fa94f6e70eeb4f41ab862270bf50b96c P 1a52d40921fe4d9da6e7adc92299989c: Bootstrap starting.
I20260811 11:59:25.860224 27139 tablet_bootstrap.cc:492] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd: Bootstrap starting.
I20260811 11:59:25.860431 27137 tablet_bootstrap.cc:654] T fa94f6e70eeb4f41ab862270bf50b96c P 1a52d40921fe4d9da6e7adc92299989c: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:25.860783 27136 tablet_bootstrap.cc:654] T fa94f6e70eeb4f41ab862270bf50b96c P af05ba16c4d24be893f3a4d151749ecb: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:25.861804 27137 log.cc:824] T fa94f6e70eeb4f41ab862270bf50b96c P 1a52d40921fe4d9da6e7adc92299989c: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:25.862023 27136 log.cc:824] T fa94f6e70eeb4f41ab862270bf50b96c P af05ba16c4d24be893f3a4d151749ecb: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:25.862545 27139 tablet_bootstrap.cc:654] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:25.863360 27139 log.cc:824] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:25.863941 27137 tablet_bootstrap.cc:492] T fa94f6e70eeb4f41ab862270bf50b96c P 1a52d40921fe4d9da6e7adc92299989c: No bootstrap required, opened a new log
I20260811 11:59:25.863942 27136 tablet_bootstrap.cc:492] T fa94f6e70eeb4f41ab862270bf50b96c P af05ba16c4d24be893f3a4d151749ecb: No bootstrap required, opened a new log
I20260811 11:59:25.864137 27136 ts_tablet_manager.cc:1397] T fa94f6e70eeb4f41ab862270bf50b96c P af05ba16c4d24be893f3a4d151749ecb: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 11:59:25.864137 27137 ts_tablet_manager.cc:1397] T fa94f6e70eeb4f41ab862270bf50b96c P 1a52d40921fe4d9da6e7adc92299989c: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260811 11:59:25.865121 27139 tablet_bootstrap.cc:492] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd: No bootstrap required, opened a new log
I20260811 11:59:25.865343 27139 ts_tablet_manager.cc:1397] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 11:59:25.868043 27139 raft_consensus.cc:348] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2158f634215c484e9f5eb7c1fb8c4fbd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42815 } } peers { permanent_uuid: "1a52d40921fe4d9da6e7adc92299989c" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43413 } } peers { permanent_uuid: "af05ba16c4d24be893f3a4d151749ecb" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 36603 } }
I20260811 11:59:25.868028 27136 raft_consensus.cc:348] T fa94f6e70eeb4f41ab862270bf50b96c P af05ba16c4d24be893f3a4d151749ecb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2158f634215c484e9f5eb7c1fb8c4fbd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42815 } } peers { permanent_uuid: "1a52d40921fe4d9da6e7adc92299989c" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43413 } } peers { permanent_uuid: "af05ba16c4d24be893f3a4d151749ecb" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 36603 } }
I20260811 11:59:25.868028 27137 raft_consensus.cc:348] T fa94f6e70eeb4f41ab862270bf50b96c P 1a52d40921fe4d9da6e7adc92299989c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2158f634215c484e9f5eb7c1fb8c4fbd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42815 } } peers { permanent_uuid: "1a52d40921fe4d9da6e7adc92299989c" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43413 } } peers { permanent_uuid: "af05ba16c4d24be893f3a4d151749ecb" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 36603 } }
I20260811 11:59:25.868295 27136 raft_consensus.cc:374] T fa94f6e70eeb4f41ab862270bf50b96c P af05ba16c4d24be893f3a4d151749ecb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:25.868295 27137 raft_consensus.cc:374] T fa94f6e70eeb4f41ab862270bf50b96c P 1a52d40921fe4d9da6e7adc92299989c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:25.868297 27139 raft_consensus.cc:374] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:25.868356 27139 raft_consensus.cc:729] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2158f634215c484e9f5eb7c1fb8c4fbd, State: Initialized, Role: FOLLOWER
I20260811 11:59:25.868356 27136 raft_consensus.cc:729] T fa94f6e70eeb4f41ab862270bf50b96c P af05ba16c4d24be893f3a4d151749ecb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af05ba16c4d24be893f3a4d151749ecb, State: Initialized, Role: FOLLOWER
I20260811 11:59:25.868356 27137 raft_consensus.cc:729] T fa94f6e70eeb4f41ab862270bf50b96c P 1a52d40921fe4d9da6e7adc92299989c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a52d40921fe4d9da6e7adc92299989c, State: Initialized, Role: FOLLOWER
I20260811 11:59:25.868806 27139 consensus_queue.cc:260] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2158f634215c484e9f5eb7c1fb8c4fbd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42815 } } peers { permanent_uuid: "1a52d40921fe4d9da6e7adc92299989c" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43413 } } peers { permanent_uuid: "af05ba16c4d24be893f3a4d151749ecb" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 36603 } }
I20260811 11:59:25.868824 27136 consensus_queue.cc:260] T fa94f6e70eeb4f41ab862270bf50b96c P af05ba16c4d24be893f3a4d151749ecb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2158f634215c484e9f5eb7c1fb8c4fbd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42815 } } peers { permanent_uuid: "1a52d40921fe4d9da6e7adc92299989c" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43413 } } peers { permanent_uuid: "af05ba16c4d24be893f3a4d151749ecb" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 36603 } }
I20260811 11:59:25.868824 27137 consensus_queue.cc:260] T fa94f6e70eeb4f41ab862270bf50b96c P 1a52d40921fe4d9da6e7adc92299989c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2158f634215c484e9f5eb7c1fb8c4fbd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42815 } } peers { permanent_uuid: "1a52d40921fe4d9da6e7adc92299989c" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43413 } } peers { permanent_uuid: "af05ba16c4d24be893f3a4d151749ecb" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 36603 } }
I20260811 11:59:25.869804 27136 ts_tablet_manager.cc:1428] T fa94f6e70eeb4f41ab862270bf50b96c P af05ba16c4d24be893f3a4d151749ecb: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 11:59:25.869820 27139 ts_tablet_manager.cc:1428] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260811 11:59:25.869839 27137 ts_tablet_manager.cc:1428] T fa94f6e70eeb4f41ab862270bf50b96c P 1a52d40921fe4d9da6e7adc92299989c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 11:59:25.870179 26977 heartbeater.cc:503] Master 127.6.143.190:35657 was elected leader, sending a full tablet report...
I20260811 11:59:25.870519 27118 heartbeater.cc:503] Master 127.6.143.190:35657 was elected leader, sending a full tablet report...
I20260811 11:59:25.871409 26833 heartbeater.cc:503] Master 127.6.143.190:35657 was elected leader, sending a full tablet report...
I20260811 11:59:26.005295 27145 raft_consensus.cc:482] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:59:26.005496 27145 raft_consensus.cc:504] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2158f634215c484e9f5eb7c1fb8c4fbd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42815 } } peers { permanent_uuid: "1a52d40921fe4d9da6e7adc92299989c" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43413 } } peers { permanent_uuid: "af05ba16c4d24be893f3a4d151749ecb" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 36603 } }
I20260811 11:59:26.006628 27145 leader_election.cc:288] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1a52d40921fe4d9da6e7adc92299989c (127.6.143.131:43413), af05ba16c4d24be893f3a4d151749ecb (127.6.143.129:36603)
I20260811 11:59:26.010798 26979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:26.006888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:43413 (local address 127.6.143.130:56629)
0811 11:59:26.007031 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:26.007048 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:26.007152 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:26.007522 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:26.007525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:26.007547 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:26.007688 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:26.007695 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:26.008615 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:26.008618 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:26.009584 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:26.009594 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:26.010329 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:26.010333 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:26.010462 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:26.010483 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 11:59:26.010562 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 11:59:26.010637 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55,"mutex_wait_us":43}
I20260811 11:59:26.010864 27135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:26.007037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:56629 (local address 127.6.143.131:43413)
0811 11:59:26.007180 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:26.007185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:26.007203 (+ 18us) server_negotiation.cc:407] Connection header received
0811 11:59:26.007267 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:26.007270 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:26.007322 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:26.007432 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:26.007829 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:26.008507 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:26.009717 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:26.010212 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:26.010306 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:26.010344 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:26.010596 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 11:59:26.010704 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 11:59:26.010910 27134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:26.007355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:56979 (local address 127.6.143.129:36603)
0811 11:59:26.007494 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:26.007499 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:26.007683 (+ 184us) server_negotiation.cc:407] Connection header received
0811 11:59:26.007750 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:26.007755 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:26.007821 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:26.007879 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:26.008215 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:26.008759 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:26.009907 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:26.010326 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:26.010440 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:26.010485 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:26.010711 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 11:59:26.010775 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 11:59:26.010924 27147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:26.007210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:36603 (local address 127.6.143.130:56979)
0811 11:59:26.007581 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:26.007592 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:26.007678 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:26.007935 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:26.007937 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:26.007963 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:26.008099 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:26.008106 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:26.008870 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:26.008873 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:26.009770 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:26.009781 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:26.010444 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:26.010447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:26.010526 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:26.010575 (+ 49us) client_negotiation.cc:769] Sending connection context
0811 11:59:26.010669 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 11:59:26.010728 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":21,"thread_start_us":115,"threads_started":1}
I20260811 11:59:26.011492 26782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa94f6e70eeb4f41ab862270bf50b96c" candidate_uuid: "2158f634215c484e9f5eb7c1fb8c4fbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af05ba16c4d24be893f3a4d151749ecb" is_pre_election: true
I20260811 11:59:26.011497 27070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa94f6e70eeb4f41ab862270bf50b96c" candidate_uuid: "2158f634215c484e9f5eb7c1fb8c4fbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a52d40921fe4d9da6e7adc92299989c" is_pre_election: true
I20260811 11:59:26.011790 26782 raft_consensus.cc:2454] T fa94f6e70eeb4f41ab862270bf50b96c P af05ba16c4d24be893f3a4d151749ecb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2158f634215c484e9f5eb7c1fb8c4fbd in term 0.
I20260811 11:59:26.011813 27070 raft_consensus.cc:2454] T fa94f6e70eeb4f41ab862270bf50b96c P 1a52d40921fe4d9da6e7adc92299989c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2158f634215c484e9f5eb7c1fb8c4fbd in term 0.
I20260811 11:59:26.012351 26856 leader_election.cc:302] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a52d40921fe4d9da6e7adc92299989c, 2158f634215c484e9f5eb7c1fb8c4fbd; no voters:
I20260811 11:59:26.012614 27145 raft_consensus.cc:2781] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:59:26.012699 27145 raft_consensus.cc:482] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:59:26.012743 27145 raft_consensus.cc:3037] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:26.013808 27145 raft_consensus.cc:504] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2158f634215c484e9f5eb7c1fb8c4fbd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42815 } } peers { permanent_uuid: "1a52d40921fe4d9da6e7adc92299989c" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43413 } } peers { permanent_uuid: "af05ba16c4d24be893f3a4d151749ecb" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 36603 } }
I20260811 11:59:26.014127 27145 leader_election.cc:288] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd [CANDIDATE]: Term 1 election: Requested vote from peers 1a52d40921fe4d9da6e7adc92299989c (127.6.143.131:43413), af05ba16c4d24be893f3a4d151749ecb (127.6.143.129:36603)
I20260811 11:59:26.014483 27070 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa94f6e70eeb4f41ab862270bf50b96c" candidate_uuid: "2158f634215c484e9f5eb7c1fb8c4fbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1a52d40921fe4d9da6e7adc92299989c"
I20260811 11:59:26.014539 26782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa94f6e70eeb4f41ab862270bf50b96c" candidate_uuid: "2158f634215c484e9f5eb7c1fb8c4fbd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af05ba16c4d24be893f3a4d151749ecb"
I20260811 11:59:26.014628 27070 raft_consensus.cc:3037] T fa94f6e70eeb4f41ab862270bf50b96c P 1a52d40921fe4d9da6e7adc92299989c [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:26.014655 26782 raft_consensus.cc:3037] T fa94f6e70eeb4f41ab862270bf50b96c P af05ba16c4d24be893f3a4d151749ecb [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:26.015645 27070 raft_consensus.cc:2454] T fa94f6e70eeb4f41ab862270bf50b96c P 1a52d40921fe4d9da6e7adc92299989c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2158f634215c484e9f5eb7c1fb8c4fbd in term 1.
I20260811 11:59:26.015671 26782 raft_consensus.cc:2454] T fa94f6e70eeb4f41ab862270bf50b96c P af05ba16c4d24be893f3a4d151749ecb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2158f634215c484e9f5eb7c1fb8c4fbd in term 1.
I20260811 11:59:26.016008 26856 leader_election.cc:302] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2158f634215c484e9f5eb7c1fb8c4fbd, af05ba16c4d24be893f3a4d151749ecb; no voters:
I20260811 11:59:26.016187 27145 raft_consensus.cc:2781] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:26.016430 27145 raft_consensus.cc:686] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd [term 1 LEADER]: Becoming Leader. State: Replica: 2158f634215c484e9f5eb7c1fb8c4fbd, State: Running, Role: LEADER
I20260811 11:59:26.016806 27145 consensus_queue.cc:237] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2158f634215c484e9f5eb7c1fb8c4fbd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42815 } } peers { permanent_uuid: "1a52d40921fe4d9da6e7adc92299989c" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43413 } } peers { permanent_uuid: "af05ba16c4d24be893f3a4d151749ecb" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 36603 } }
I20260811 11:59:26.020148 26622 catalog_manager.cc:5495] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd reported cstate change: term changed from 0 to 1, leader changed from <none> to 2158f634215c484e9f5eb7c1fb8c4fbd (127.6.143.130). New cstate: current_term: 1 leader_uuid: "2158f634215c484e9f5eb7c1fb8c4fbd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2158f634215c484e9f5eb7c1fb8c4fbd" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 42815 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1a52d40921fe4d9da6e7adc92299989c" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43413 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af05ba16c4d24be893f3a4d151749ecb" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 36603 } health_report { overall_health: UNKNOWN } } }
I20260811 11:59:26.035043 27132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:26.031222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57146 (local address 127.6.143.130:42815)
0811 11:59:26.031346 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:26.031352 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:26.031369 (+ 17us) server_negotiation.cc:407] Connection header received
0811 11:59:26.031567 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:26.031569 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:26.031640 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:26.031749 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:59:26.032065 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:26.032611 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:26.033460 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:26.033741 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:26.033853 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:26.034186 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:59:26.034585 (+ 399us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:59:26.034748 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:26.034846 (+ 98us) server_negotiation.cc:299] Negotiation successful
0811 11:59:26.034883 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 11:59:26.040828 26782 raft_consensus.cc:1261] T fa94f6e70eeb4f41ab862270bf50b96c P af05ba16c4d24be893f3a4d151749ecb [term 1 FOLLOWER]: Refusing update from remote peer 2158f634215c484e9f5eb7c1fb8c4fbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:59:26.040828 27070 raft_consensus.cc:1261] T fa94f6e70eeb4f41ab862270bf50b96c P 1a52d40921fe4d9da6e7adc92299989c [term 1 FOLLOWER]: Refusing update from remote peer 2158f634215c484e9f5eb7c1fb8c4fbd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:59:26.041538 27145 consensus_queue.cc:1035] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "af05ba16c4d24be893f3a4d151749ecb" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 36603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:59:26.041764 27148 consensus_queue.cc:1035] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd [LEADER]: Connected to new peer: Peer: permanent_uuid: "1a52d40921fe4d9da6e7adc92299989c" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 43413 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 11:59:26.048027 27119 tablet.cc:2369] T fa94f6e70eeb4f41ab862270bf50b96c P 1a52d40921fe4d9da6e7adc92299989c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:59:26.059609 26624 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:59:26.059767 26624 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47880:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:59:26.062261 26624 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f612d51471d9435a85900932a2158369: Sending DeleteTablet for 3 replicas of tablet fa94f6e70eeb4f41ab862270bf50b96c
I20260811 11:59:26.063162 26905 tablet_service.cc:1514] Processing DeleteTablet for tablet fa94f6e70eeb4f41ab862270bf50b96c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:59:26 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:57144
I20260811 11:59:26.063606 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 26690
I20260811 11:59:26.063684 27047 tablet_service.cc:1514] Processing DeleteTablet for tablet fa94f6e70eeb4f41ab862270bf50b96c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:59:26 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:58964
I20260811 11:59:26.068101 27163 tablet_replica.cc:330] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd: stopping tablet replica
I20260811 11:59:26.068413 27163 raft_consensus.cc:2229] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:59:26.068794 27163 raft_consensus.cc:2258] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:59:26.070318 27163 ts_tablet_manager.cc:1905] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:59:26.073071 27163 ts_tablet_manager.cc:1918] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:59:26.073171 27163 log.cc:1196] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-1/wal/wals/fa94f6e70eeb4f41ab862270bf50b96c
I20260811 11:59:26.073513 27163 ts_tablet_manager.cc:1939] T fa94f6e70eeb4f41ab862270bf50b96c P 2158f634215c484e9f5eb7c1fb8c4fbd: Deleting consensus metadata
I20260811 11:59:26.074801 26609 catalog_manager.cc:4841] TS 2158f634215c484e9f5eb7c1fb8c4fbd (127.6.143.130:42815): tablet fa94f6e70eeb4f41ab862270bf50b96c (table test-table [id=0013118e9619462c9011e3da64bc6bdd]) successfully deleted
W20260811 11:59:26.075160 26606 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.143.129:36603 (error 108)
I20260811 11:59:26.075417 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 26838
I20260811 11:59:26.078227 27133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:26.077893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:36603 (local address 127.0.0.1:53836)
0811 11:59:26.078047 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:26.078124 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.129:36603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 11:59:26.078387 26606 catalog_manager.cc:4584] TS af05ba16c4d24be893f3a4d151749ecb (127.6.143.129:36603): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fa94f6e70eeb4f41ab862270bf50b96c: Network error: Client connection negotiation failed: client connection to 127.6.143.129:36603: connect: Connection refused (error 111)
I20260811 11:59:26.079643 27165 tablet_replica.cc:330] T fa94f6e70eeb4f41ab862270bf50b96c P 1a52d40921fe4d9da6e7adc92299989c: stopping tablet replica
I20260811 11:59:26.080000 27165 raft_consensus.cc:2229] T fa94f6e70eeb4f41ab862270bf50b96c P 1a52d40921fe4d9da6e7adc92299989c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:59:26.080397 27165 raft_consensus.cc:2258] T fa94f6e70eeb4f41ab862270bf50b96c P 1a52d40921fe4d9da6e7adc92299989c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:59:26.082118 27165 ts_tablet_manager.cc:1905] T fa94f6e70eeb4f41ab862270bf50b96c P 1a52d40921fe4d9da6e7adc92299989c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:59:26.085623 27165 ts_tablet_manager.cc:1918] T fa94f6e70eeb4f41ab862270bf50b96c P 1a52d40921fe4d9da6e7adc92299989c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:59:26.085732 27165 log.cc:1196] T fa94f6e70eeb4f41ab862270bf50b96c P 1a52d40921fe4d9da6e7adc92299989c: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786449319379472-6718-0/minicluster-data/ts-2/wal/wals/fa94f6e70eeb4f41ab862270bf50b96c
I20260811 11:59:26.086071 27165 ts_tablet_manager.cc:1939] T fa94f6e70eeb4f41ab862270bf50b96c P 1a52d40921fe4d9da6e7adc92299989c: Deleting consensus metadata
I20260811 11:59:26.087446 26606 catalog_manager.cc:4841] TS 1a52d40921fe4d9da6e7adc92299989c (127.6.143.131:43413): tablet fa94f6e70eeb4f41ab862270bf50b96c (table test-table [id=0013118e9619462c9011e3da64bc6bdd]) successfully deleted
I20260811 11:59:26.087997 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 26982
I20260811 11:59:26.098047 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 26587
2026-08-11T11:59:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2447 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T11:59:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:59:26Z Disabled control of system clock
I20260811 11:59:26.131001 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:41639
--webserver_interface=127.6.143.190
--webserver_port=0
--builtin_ntp_servers=127.6.143.148:41657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:41639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 11:59:26.242694 27170 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:26.242941 27170 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:26.242975 27170 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:26.246959 27170 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:59:26.247013 27170 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:26.247030 27170 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:26.247045 27170 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:59:26.247061 27170 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:59:26.251609 27170 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:41657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:41639
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:41639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27170
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:26.252616 27170 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:26.253919 27170 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:26.262372 27176 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:26.262408 27177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:26.262557 27170 server_base.cc:1034] running on GCE node
W20260811 11:59:26.262669 27179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:26.263247 27170 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:26.264393 27170 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:26.265589 27170 hybrid_clock.cc:648] HybridClock initialized: now 1786449566265577 us; error 27 us; skew 500 ppm
I20260811 11:59:26.267267 27170 webserver.cc:460] Webserver started at http://127.6.143.190:39011/ using document root <none> and password file <none>
I20260811 11:59:26.267838 27170 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:26.267956 27170 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:26.268195 27170 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:26.270140 27170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "3347eef1c6cd41f0b5519fa58753fbf0"
format_stamp: "Formatted at 2026-08-11 11:59:26 on dist-test-slave-zkpd"
server_key: "f7b86233562c29471f96866529f5879e"
server_key_iv: "8b8702c9e8dbb4b23a5eb3b344e2017c"
server_key_version: "encryptionkey@0"
I20260811 11:59:26.270702 27170 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "3347eef1c6cd41f0b5519fa58753fbf0"
format_stamp: "Formatted at 2026-08-11 11:59:26 on dist-test-slave-zkpd"
server_key: "f7b86233562c29471f96866529f5879e"
server_key_iv: "8b8702c9e8dbb4b23a5eb3b344e2017c"
server_key_version: "encryptionkey@0"
I20260811 11:59:26.274252 27170 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 11:59:26.276558 27186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:26.277779 27170 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:59:26.277920 27170 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "3347eef1c6cd41f0b5519fa58753fbf0"
format_stamp: "Formatted at 2026-08-11 11:59:26 on dist-test-slave-zkpd"
server_key: "f7b86233562c29471f96866529f5879e"
server_key_iv: "8b8702c9e8dbb4b23a5eb3b344e2017c"
server_key_version: "encryptionkey@0"
I20260811 11:59:26.278026 27170 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:26.290819 27170 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:26.291441 27170 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:26.291630 27170 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:26.298359 27170 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:41639
I20260811 11:59:26.298446 27243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:41639 every 8 connection(s)
I20260811 11:59:26.299301 27170 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:59:26.302628 27244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:26.306741 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 27170
I20260811 11:59:26.306885 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:59:26.307214 6718 external_mini_cluster.cc:1428] Setting key dd9248197c06036d35bcac4f03dfadb4
I20260811 11:59:26.309130 27244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3347eef1c6cd41f0b5519fa58753fbf0: Bootstrap starting.
I20260811 11:59:26.311323 27244 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3347eef1c6cd41f0b5519fa58753fbf0: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:26.312155 27244 log.cc:824] T 00000000000000000000000000000000 P 3347eef1c6cd41f0b5519fa58753fbf0: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:26.314085 27244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3347eef1c6cd41f0b5519fa58753fbf0: No bootstrap required, opened a new log
I20260811 11:59:26.316689 27247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:26.308735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60850 (local address 127.6.143.190:41639)
0811 11:59:26.309411 (+ 676us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:26.309429 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:26.309465 (+ 36us) server_negotiation.cc:407] Connection header received
0811 11:59:26.310093 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:26.310108 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:26.310498 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:26.310830 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:26.311272 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:26.312365 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:26.313243 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:26.313721 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:26.313895 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:26.314817 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:26.314821 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:26.314834 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:26.315203 (+ 369us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:26.315259 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:26.315278 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:26.315388 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:26.315540 (+ 152us) server_negotiation.cc:299] Negotiation successful
0811 11:59:26.315781 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":123,"threads_started":1}
I20260811 11:59:26.317343 27244 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3347eef1c6cd41f0b5519fa58753fbf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3347eef1c6cd41f0b5519fa58753fbf0" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 41639 } }
I20260811 11:59:26.317569 27244 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3347eef1c6cd41f0b5519fa58753fbf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:26.317627 27244 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3347eef1c6cd41f0b5519fa58753fbf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3347eef1c6cd41f0b5519fa58753fbf0, State: Initialized, Role: FOLLOWER
I20260811 11:59:26.318086 27244 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3347eef1c6cd41f0b5519fa58753fbf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3347eef1c6cd41f0b5519fa58753fbf0" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 41639 } }
I20260811 11:59:26.318239 27244 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3347eef1c6cd41f0b5519fa58753fbf0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:59:26.318331 27244 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3347eef1c6cd41f0b5519fa58753fbf0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:59:26.318451 27244 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3347eef1c6cd41f0b5519fa58753fbf0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:26.319366 27244 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3347eef1c6cd41f0b5519fa58753fbf0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3347eef1c6cd41f0b5519fa58753fbf0" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 41639 } }
I20260811 11:59:26.319700 27244 leader_election.cc:302] T 00000000000000000000000000000000 P 3347eef1c6cd41f0b5519fa58753fbf0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3347eef1c6cd41f0b5519fa58753fbf0; no voters:
I20260811 11:59:26.320048 27244 leader_election.cc:288] T 00000000000000000000000000000000 P 3347eef1c6cd41f0b5519fa58753fbf0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:59:26.320147 27250 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3347eef1c6cd41f0b5519fa58753fbf0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:26.320415 27250 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3347eef1c6cd41f0b5519fa58753fbf0 [term 1 LEADER]: Becoming Leader. State: Replica: 3347eef1c6cd41f0b5519fa58753fbf0, State: Running, Role: LEADER
I20260811 11:59:26.320716 27250 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3347eef1c6cd41f0b5519fa58753fbf0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3347eef1c6cd41f0b5519fa58753fbf0" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 41639 } }
I20260811 11:59:26.321286 27244 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3347eef1c6cd41f0b5519fa58753fbf0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:59:26.322378 27251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3347eef1c6cd41f0b5519fa58753fbf0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3347eef1c6cd41f0b5519fa58753fbf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3347eef1c6cd41f0b5519fa58753fbf0" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 41639 } } }
I20260811 11:59:26.322530 27251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3347eef1c6cd41f0b5519fa58753fbf0 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:26.322839 27252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3347eef1c6cd41f0b5519fa58753fbf0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3347eef1c6cd41f0b5519fa58753fbf0. Latest consensus state: current_term: 1 leader_uuid: "3347eef1c6cd41f0b5519fa58753fbf0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3347eef1c6cd41f0b5519fa58753fbf0" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 41639 } } }
I20260811 11:59:26.322934 27252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3347eef1c6cd41f0b5519fa58753fbf0 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:26.323051 27259 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:59:26.326071 27259 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:59:26.331436 27259 catalog_manager.cc:1347] Generated new cluster ID: 49073eacf0ea404baedd85da72c0452a
I20260811 11:59:26.331509 27259 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:59:26.362377 27259 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:59:26.363601 27259 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:59:26.369879 27259 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3347eef1c6cd41f0b5519fa58753fbf0: Generated new TSK 0
I20260811 11:59:26.370564 27259 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:59:26.379606 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:0
--local_ip_for_outbound_sockets=127.6.143.129
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:41639
--builtin_ntp_servers=127.6.143.148:41657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 11:59:26.509354 27269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:26.509584 27269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:26.509656 27269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:26.513664 27269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:26.513734 27269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:26.513831 27269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:59:26.518383 27269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:41657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:41639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:26.519353 27269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:26.520738 27269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:26.530573 27277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:26.530580 27275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:26.530580 27274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:26.530845 27269 server_base.cc:1034] running on GCE node
I20260811 11:59:26.531307 27269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:26.531972 27269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:26.533160 27269 hybrid_clock.cc:648] HybridClock initialized: now 1786449566533147 us; error 32 us; skew 500 ppm
I20260811 11:59:26.535000 27269 webserver.cc:460] Webserver started at http://127.6.143.129:39707/ using document root <none> and password file <none>
I20260811 11:59:26.535565 27269 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:26.535657 27269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:26.535878 27269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:26.537992 27269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-0/data/instance:
uuid: "279549c47ec9448e8d9698eedf3d88c0"
format_stamp: "Formatted at 2026-08-11 11:59:26 on dist-test-slave-zkpd"
server_key: "9eba3a75e2604d73cf7074962ff9aa62"
server_key_iv: "fc83e79465a44860124a952cbdf08b22"
server_key_version: "encryptionkey@0"
I20260811 11:59:26.538569 27269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance:
uuid: "279549c47ec9448e8d9698eedf3d88c0"
format_stamp: "Formatted at 2026-08-11 11:59:26 on dist-test-slave-zkpd"
server_key: "9eba3a75e2604d73cf7074962ff9aa62"
server_key_iv: "fc83e79465a44860124a952cbdf08b22"
server_key_version: "encryptionkey@0"
I20260811 11:59:26.542366 27269 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260811 11:59:26.544945 27284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:26.546202 27269 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260811 11:59:26.546344 27269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "279549c47ec9448e8d9698eedf3d88c0"
format_stamp: "Formatted at 2026-08-11 11:59:26 on dist-test-slave-zkpd"
server_key: "9eba3a75e2604d73cf7074962ff9aa62"
server_key_iv: "fc83e79465a44860124a952cbdf08b22"
server_key_version: "encryptionkey@0"
I20260811 11:59:26.546461 27269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:26.567430 27269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:26.568317 27269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:26.568526 27269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:26.569164 27269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:59:26.570154 27269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:59:26.570210 27269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:26.570281 27269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:59:26.570310 27269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:26.579255 27269 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:39691
I20260811 11:59:26.579283 27405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:39691 every 8 connection(s)
I20260811 11:59:26.580222 27269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:59:26.588104 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 27269
I20260811 11:59:26.588253 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance
I20260811 11:59:26.588629 6718 external_mini_cluster.cc:1428] Setting key b490105fc84a6759e55a5ebc05d38048
I20260811 11:59:26.589745 27247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:26.582184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:34093 (local address 127.6.143.190:41639)
0811 11:59:26.582336 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:26.582341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:26.583497 (+ 1156us) server_negotiation.cc:407] Connection header received
0811 11:59:26.584904 (+ 1407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:26.584906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:26.584953 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:26.585039 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:26.585894 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:26.586429 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:26.587509 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:26.587738 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:26.587846 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:26.589187 (+ 1341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:26.589190 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:26.589192 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:26.589204 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:26.589224 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:26.589232 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:26.589292 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:26.589522 (+ 230us) server_negotiation.cc:299] Negotiation successful
0811 11:59:26.589598 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 11:59:26.590575 27408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:26.582663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:41639 (local address 127.6.143.129:34093)
0811 11:59:26.583341 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:26.583380 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:26.584700 (+ 1320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:26.585205 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:26.585211 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:26.585289 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:59:26.585773 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:26.585784 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:26.586538 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:26.586540 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:26.587401 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:26.587411 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:26.587925 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:26.587932 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:26.588107 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:26.588872 (+ 765us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:59:26.588874 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:59:26.588916 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:59:26.588978 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:59:26.589321 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:59:26.589331 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:59:26.589338 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 11:59:26.589482 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 11:59:26.589699 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":133,"threads_started":1}
I20260811 11:59:26.590961 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:0
--local_ip_for_outbound_sockets=127.6.143.130
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:41639
--builtin_ntp_servers=127.6.143.148:41657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 11:59:26.591814 27406 heartbeater.cc:347] Connected to a master server at 127.6.143.190:41639
I20260811 11:59:26.592082 27406 heartbeater.cc:464] Registering TS with master...
I20260811 11:59:26.592581 27406 heartbeater.cc:511] Master 127.6.143.190:41639 requested a full tablet report, sending...
I20260811 11:59:26.594491 27204 ts_manager.cc:194] Registered new tserver with Master: 279549c47ec9448e8d9698eedf3d88c0 (127.6.143.129:39691)
I20260811 11:59:26.596374 27204 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.143.129:34093
W20260811 11:59:26.710618 27409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:26.710865 27409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:26.710901 27409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:26.714820 27409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:26.714872 27409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:26.714923 27409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:59:26.719396 27409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:41657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:41639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:26.720398 27409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:26.721722 27409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:26.731227 27415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:26.731220 27414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:26.731204 27417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:26.731485 27409 server_base.cc:1034] running on GCE node
I20260811 11:59:26.732040 27409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:26.732589 27409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:26.733744 27409 hybrid_clock.cc:648] HybridClock initialized: now 1786449566733715 us; error 42 us; skew 500 ppm
I20260811 11:59:26.735510 27409 webserver.cc:460] Webserver started at http://127.6.143.130:38347/ using document root <none> and password file <none>
I20260811 11:59:26.736135 27409 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:26.736198 27409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:26.736406 27409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:26.738337 27409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-1/data/instance:
uuid: "16eeeadf08c54649ad1271be06ce250a"
format_stamp: "Formatted at 2026-08-11 11:59:26 on dist-test-slave-zkpd"
server_key: "21ce53dd9bcd7eb88daf88495ee322d0"
server_key_iv: "1f42c1724d9009755921eabce1addc3c"
server_key_version: "encryptionkey@0"
I20260811 11:59:26.738909 27409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance:
uuid: "16eeeadf08c54649ad1271be06ce250a"
format_stamp: "Formatted at 2026-08-11 11:59:26 on dist-test-slave-zkpd"
server_key: "21ce53dd9bcd7eb88daf88495ee322d0"
server_key_iv: "1f42c1724d9009755921eabce1addc3c"
server_key_version: "encryptionkey@0"
I20260811 11:59:26.742440 27409 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 11:59:26.744783 27426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:26.745927 27409 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:59:26.746071 27409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "16eeeadf08c54649ad1271be06ce250a"
format_stamp: "Formatted at 2026-08-11 11:59:26 on dist-test-slave-zkpd"
server_key: "21ce53dd9bcd7eb88daf88495ee322d0"
server_key_iv: "1f42c1724d9009755921eabce1addc3c"
server_key_version: "encryptionkey@0"
I20260811 11:59:26.746198 27409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:26.776667 27409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:26.777346 27409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:26.777554 27409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:26.778141 27409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:59:26.779071 27409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:59:26.779145 27409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:26.779217 27409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:59:26.779259 27409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:26.788139 27409 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:41671
I20260811 11:59:26.788218 27552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:41671 every 8 connection(s)
I20260811 11:59:26.789037 27409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
I20260811 11:59:26.798014 27247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:26.790841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:38199 (local address 127.6.143.190:41639)
0811 11:59:26.790995 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:26.791000 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:26.792161 (+ 1161us) server_negotiation.cc:407] Connection header received
0811 11:59:26.793532 (+ 1371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:26.793535 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:26.793584 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:26.793660 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:26.794607 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:26.795162 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:26.796326 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:26.796529 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:26.796609 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:26.797604 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:26.797607 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:26.797609 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:26.797621 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:26.797640 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:26.797648 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:26.797715 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:26.797852 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 11:59:26.797900 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 11:59:26.798810 27555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:26.791308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:41639 (local address 127.6.143.130:38199)
0811 11:59:26.792007 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:26.792049 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:26.793329 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:26.793869 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:26.793876 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:26.793972 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:59:26.794459 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:26.794472 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:26.795276 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:26.795279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:26.796216 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:26.796226 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:26.796662 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:26.796664 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:26.796797 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:26.797351 (+ 554us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:59:26.797353 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:59:26.797392 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:59:26.797454 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:59:26.797726 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:59:26.797733 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:59:26.797737 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 11:59:26.797830 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 11:59:26.798047 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":108,"threads_started":1}
I20260811 11:59:26.799346 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 27409
I20260811 11:59:26.799507 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance
I20260811 11:59:26.799817 6718 external_mini_cluster.cc:1428] Setting key 0be479f7b1e75492a785a26374c908fa
I20260811 11:59:26.800091 27553 heartbeater.cc:347] Connected to a master server at 127.6.143.190:41639
I20260811 11:59:26.800341 27553 heartbeater.cc:464] Registering TS with master...
I20260811 11:59:26.800884 27553 heartbeater.cc:511] Master 127.6.143.190:41639 requested a full tablet report, sending...
I20260811 11:59:26.801741 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:0
--local_ip_for_outbound_sockets=127.6.143.131
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:41639
--builtin_ntp_servers=127.6.143.148:41657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 11:59:26.801971 27204 ts_manager.cc:194] Registered new tserver with Master: 16eeeadf08c54649ad1271be06ce250a (127.6.143.130:41671)
I20260811 11:59:26.802760 27204 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.143.130:38199
W20260811 11:59:26.915913 27556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:26.916167 27556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:26.916254 27556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:26.920224 27556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:26.920286 27556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:26.920382 27556 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:59:26.925271 27556 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:41657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:41639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27556
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:26.926218 27556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:26.927500 27556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 11:59:26.936918 27556 server_base.cc:1034] running on GCE node
W20260811 11:59:26.936851 27561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:26.936873 27564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:26.936870 27562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:26.937541 27556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:26.938167 27556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:26.939371 27556 hybrid_clock.cc:648] HybridClock initialized: now 1786449566939332 us; error 47 us; skew 500 ppm
I20260811 11:59:26.941285 27556 webserver.cc:460] Webserver started at http://127.6.143.131:45717/ using document root <none> and password file <none>
I20260811 11:59:26.941859 27556 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:26.941951 27556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:26.942185 27556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:26.944276 27556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-2/data/instance:
uuid: "85aafda77b3c49a59d9ddd28b7cd9709"
format_stamp: "Formatted at 2026-08-11 11:59:26 on dist-test-slave-zkpd"
server_key: "cc4f2579517a59bbd0448d0a865b98b5"
server_key_iv: "8f6c83e505422bb9ec0819f72b84690b"
server_key_version: "encryptionkey@0"
I20260811 11:59:26.944829 27556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance:
uuid: "85aafda77b3c49a59d9ddd28b7cd9709"
format_stamp: "Formatted at 2026-08-11 11:59:26 on dist-test-slave-zkpd"
server_key: "cc4f2579517a59bbd0448d0a865b98b5"
server_key_iv: "8f6c83e505422bb9ec0819f72b84690b"
server_key_version: "encryptionkey@0"
I20260811 11:59:26.948415 27556 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 11:59:26.950795 27571 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:26.952032 27556 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:59:26.952169 27556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "85aafda77b3c49a59d9ddd28b7cd9709"
format_stamp: "Formatted at 2026-08-11 11:59:26 on dist-test-slave-zkpd"
server_key: "cc4f2579517a59bbd0448d0a865b98b5"
server_key_iv: "8f6c83e505422bb9ec0819f72b84690b"
server_key_version: "encryptionkey@0"
I20260811 11:59:26.952291 27556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:26.964581 27556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:26.965200 27556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:26.965391 27556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:26.965938 27556 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:59:26.966830 27556 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:59:26.966902 27556 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:26.966969 27556 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:59:26.967020 27556 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:26.975533 27556 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:36143
I20260811 11:59:26.975556 27693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:36143 every 8 connection(s)
I20260811 11:59:26.976394 27556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:59:26.979491 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 27556
I20260811 11:59:26.979620 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance
I20260811 11:59:26.979951 6718 external_mini_cluster.cc:1428] Setting key e6650f537b507391fa6ea720ac71b29f
I20260811 11:59:26.985715 27247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:26.978212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:57829 (local address 127.6.143.190:41639)
0811 11:59:26.978351 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:26.978354 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:26.979354 (+ 1000us) server_negotiation.cc:407] Connection header received
0811 11:59:26.980955 (+ 1601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:26.980958 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:26.981017 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:26.981097 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:26.982022 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:26.982550 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:26.983692 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:26.983990 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:26.984105 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:26.985206 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:26.985210 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:26.985213 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:26.985226 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:26.985248 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:26.985264 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:26.985342 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:26.985515 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 11:59:26.985572 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 11:59:26.986474 27696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:26.978555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:41639 (local address 127.6.143.131:57829)
0811 11:59:26.979170 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:26.979224 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:26.980724 (+ 1500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:26.981269 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:26.981276 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:26.981359 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:59:26.981876 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:26.981889 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:26.982679 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:26.982683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:26.983551 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:26.983561 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:26.984140 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:26.984144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:26.984313 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:26.984945 (+ 632us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:59:26.984947 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:59:26.984988 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:59:26.985049 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:59:26.985358 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:59:26.985369 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:59:26.985382 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 11:59:26.985509 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 11:59:26.985768 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":116,"threads_started":1}
I20260811 11:59:26.987514 27694 heartbeater.cc:347] Connected to a master server at 127.6.143.190:41639
I20260811 11:59:26.987764 27694 heartbeater.cc:464] Registering TS with master...
I20260811 11:59:26.988297 27694 heartbeater.cc:511] Master 127.6.143.190:41639 requested a full tablet report, sending...
I20260811 11:59:26.989327 27204 ts_manager.cc:194] Registered new tserver with Master: 85aafda77b3c49a59d9ddd28b7cd9709 (127.6.143.131:36143)
I20260811 11:59:26.989887 27204 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.143.131:57829
I20260811 11:59:26.993513 6718 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:59:26.999116 27247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:26.995491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60860 (local address 127.6.143.190:41639)
0811 11:59:26.995610 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:26.995613 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:26.995856 (+ 243us) server_negotiation.cc:407] Connection header received
0811 11:59:26.995994 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:26.995996 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:26.996032 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:26.996096 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:26.996374 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:26.996846 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:26.997649 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:26.997849 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:26.997930 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:26.998734 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:26.998736 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:26.998738 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:26.998748 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:26.998763 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:26.998769 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:26.998854 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:26.998990 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 11:59:26.999022 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 11:59:27.001892 27204 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60860:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:59:27.004873 27204 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:59:27.019873 27707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:27.013797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58520 (local address 127.6.143.130:41671)
0811 11:59:27.014160 (+ 363us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:27.014172 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:27.014210 (+ 38us) server_negotiation.cc:407] Connection header received
0811 11:59:27.014467 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:27.014474 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:27.014696 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:27.014868 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:27.015465 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:27.016630 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:27.018299 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:27.019086 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:27.019256 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:27.019471 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:27.019576 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 11:59:27.019676 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":126,"threads_started":1}
I20260811 11:59:27.019888 27708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:27.014436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:36143 (local address 127.0.0.1:51494)
0811 11:59:27.015146 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:27.015161 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:27.015297 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:27.015794 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:27.015797 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:27.015817 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:27.015932 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:27.015943 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:27.017192 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:27.017195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:27.018318 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:27.018326 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:27.019342 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:27.019346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:27.019438 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:27.019465 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 11:59:27.019527 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 11:59:27.019689 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":646,"mutex_wait_us":3,"thread_start_us":109,"threads_started":1}
I20260811 11:59:27.019865 27706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:27.013642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:41671 (local address 127.0.0.1:58520)
0811 11:59:27.014026 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:27.014078 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:27.014258 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:27.015114 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:27.015121 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:27.015151 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:27.015316 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:27.015325 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:27.016741 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:27.016744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:27.018152 (+ 1408us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:27.018163 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:27.019255 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:27.019258 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:27.019383 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:27.019419 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 11:59:27.019550 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 11:59:27.019687 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":34,"thread_start_us":109,"threads_started":1}
I20260811 11:59:27.020534 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:27.014648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51494 (local address 127.6.143.131:36143)
0811 11:59:27.015117 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:27.015129 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:27.015281 (+ 152us) server_negotiation.cc:407] Connection header received
0811 11:59:27.015417 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:27.015423 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:27.015635 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:27.015817 (+ 182us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:27.016036 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:27.017057 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:27.018438 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:27.019159 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:27.019990 (+ 831us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:27.020214 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:27.020267 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 11:59:27.020369 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":160,"threads_started":1}
I20260811 11:59:27.021231 27710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:27.014804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:39691 (local address 127.0.0.1:46450)
0811 11:59:27.015679 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:27.015693 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:27.015797 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:27.016334 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:27.016337 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:27.016357 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:27.016438 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:27.016443 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:27.017658 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:27.017664 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:27.018691 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:27.018729 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:27.020862 (+ 2133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:27.020866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:27.020963 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:27.020985 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 11:59:27.021050 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 11:59:27.021113 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":812,"thread_start_us":103,"threads_started":1}
I20260811 11:59:27.021952 27711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:27.015278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46450 (local address 127.6.143.129:39691)
0811 11:59:27.015613 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:27.015625 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:27.015921 (+ 296us) server_negotiation.cc:407] Connection header received
0811 11:59:27.016011 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:27.016016 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:27.016185 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:27.016328 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:27.016556 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:27.017498 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:27.018865 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:27.019743 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:27.019845 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:27.021597 (+ 1752us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:27.021648 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 11:59:27.021744 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":112,"threads_started":1}
I20260811 11:59:27.023236 27480 tablet_service.cc:1467] Processing CreateTablet for tablet 52b0317878394561b929a5c31ad340cc (DEFAULT_TABLE table=test-table [id=ca8a0d42ccd141f9b2ac88f8f005c12c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:59:27.023581 27622 tablet_service.cc:1467] Processing CreateTablet for tablet 52b0317878394561b929a5c31ad340cc (DEFAULT_TABLE table=test-table [id=ca8a0d42ccd141f9b2ac88f8f005c12c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:59:27.024375 27480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52b0317878394561b929a5c31ad340cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:27.024652 27622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52b0317878394561b929a5c31ad340cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:27.024653 27332 tablet_service.cc:1467] Processing CreateTablet for tablet 52b0317878394561b929a5c31ad340cc (DEFAULT_TABLE table=test-table [id=ca8a0d42ccd141f9b2ac88f8f005c12c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:59:27.025655 27332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52b0317878394561b929a5c31ad340cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:27.030658 27714 tablet_bootstrap.cc:492] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a: Bootstrap starting.
I20260811 11:59:27.032056 27715 tablet_bootstrap.cc:492] T 52b0317878394561b929a5c31ad340cc P 85aafda77b3c49a59d9ddd28b7cd9709: Bootstrap starting.
I20260811 11:59:27.033107 27716 tablet_bootstrap.cc:492] T 52b0317878394561b929a5c31ad340cc P 279549c47ec9448e8d9698eedf3d88c0: Bootstrap starting.
I20260811 11:59:27.034276 27715 tablet_bootstrap.cc:654] T 52b0317878394561b929a5c31ad340cc P 85aafda77b3c49a59d9ddd28b7cd9709: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:27.034276 27714 tablet_bootstrap.cc:654] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:27.034875 27716 tablet_bootstrap.cc:654] T 52b0317878394561b929a5c31ad340cc P 279549c47ec9448e8d9698eedf3d88c0: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:27.035181 27714 log.cc:824] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:27.035181 27715 log.cc:824] T 52b0317878394561b929a5c31ad340cc P 85aafda77b3c49a59d9ddd28b7cd9709: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:27.035669 27716 log.cc:824] T 52b0317878394561b929a5c31ad340cc P 279549c47ec9448e8d9698eedf3d88c0: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:27.037125 27714 tablet_bootstrap.cc:492] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a: No bootstrap required, opened a new log
I20260811 11:59:27.037261 27715 tablet_bootstrap.cc:492] T 52b0317878394561b929a5c31ad340cc P 85aafda77b3c49a59d9ddd28b7cd9709: No bootstrap required, opened a new log
I20260811 11:59:27.037382 27714 ts_tablet_manager.cc:1397] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 11:59:27.037449 27715 ts_tablet_manager.cc:1397] T 52b0317878394561b929a5c31ad340cc P 85aafda77b3c49a59d9ddd28b7cd9709: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 11:59:27.037541 27716 tablet_bootstrap.cc:492] T 52b0317878394561b929a5c31ad340cc P 279549c47ec9448e8d9698eedf3d88c0: No bootstrap required, opened a new log
I20260811 11:59:27.037717 27716 ts_tablet_manager.cc:1397] T 52b0317878394561b929a5c31ad340cc P 279549c47ec9448e8d9698eedf3d88c0: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 11:59:27.040984 27714 raft_consensus.cc:348] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16eeeadf08c54649ad1271be06ce250a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41671 } } peers { permanent_uuid: "85aafda77b3c49a59d9ddd28b7cd9709" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36143 } } peers { permanent_uuid: "279549c47ec9448e8d9698eedf3d88c0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39691 } }
I20260811 11:59:27.041235 27714 raft_consensus.cc:374] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:27.041292 27714 raft_consensus.cc:729] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16eeeadf08c54649ad1271be06ce250a, State: Initialized, Role: FOLLOWER
I20260811 11:59:27.041721 27714 consensus_queue.cc:260] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16eeeadf08c54649ad1271be06ce250a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41671 } } peers { permanent_uuid: "85aafda77b3c49a59d9ddd28b7cd9709" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36143 } } peers { permanent_uuid: "279549c47ec9448e8d9698eedf3d88c0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39691 } }
I20260811 11:59:27.041776 27716 raft_consensus.cc:348] T 52b0317878394561b929a5c31ad340cc P 279549c47ec9448e8d9698eedf3d88c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16eeeadf08c54649ad1271be06ce250a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41671 } } peers { permanent_uuid: "85aafda77b3c49a59d9ddd28b7cd9709" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36143 } } peers { permanent_uuid: "279549c47ec9448e8d9698eedf3d88c0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39691 } }
I20260811 11:59:27.041776 27715 raft_consensus.cc:348] T 52b0317878394561b929a5c31ad340cc P 85aafda77b3c49a59d9ddd28b7cd9709 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16eeeadf08c54649ad1271be06ce250a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41671 } } peers { permanent_uuid: "85aafda77b3c49a59d9ddd28b7cd9709" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36143 } } peers { permanent_uuid: "279549c47ec9448e8d9698eedf3d88c0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39691 } }
I20260811 11:59:27.042028 27716 raft_consensus.cc:374] T 52b0317878394561b929a5c31ad340cc P 279549c47ec9448e8d9698eedf3d88c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:27.042028 27715 raft_consensus.cc:374] T 52b0317878394561b929a5c31ad340cc P 85aafda77b3c49a59d9ddd28b7cd9709 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:27.042088 27716 raft_consensus.cc:729] T 52b0317878394561b929a5c31ad340cc P 279549c47ec9448e8d9698eedf3d88c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 279549c47ec9448e8d9698eedf3d88c0, State: Initialized, Role: FOLLOWER
I20260811 11:59:27.042097 27715 raft_consensus.cc:729] T 52b0317878394561b929a5c31ad340cc P 85aafda77b3c49a59d9ddd28b7cd9709 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85aafda77b3c49a59d9ddd28b7cd9709, State: Initialized, Role: FOLLOWER
I20260811 11:59:27.042481 27553 heartbeater.cc:503] Master 127.6.143.190:41639 was elected leader, sending a full tablet report...
I20260811 11:59:27.042536 27715 consensus_queue.cc:260] T 52b0317878394561b929a5c31ad340cc P 85aafda77b3c49a59d9ddd28b7cd9709 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16eeeadf08c54649ad1271be06ce250a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41671 } } peers { permanent_uuid: "85aafda77b3c49a59d9ddd28b7cd9709" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36143 } } peers { permanent_uuid: "279549c47ec9448e8d9698eedf3d88c0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39691 } }
I20260811 11:59:27.042558 27716 consensus_queue.cc:260] T 52b0317878394561b929a5c31ad340cc P 279549c47ec9448e8d9698eedf3d88c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16eeeadf08c54649ad1271be06ce250a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41671 } } peers { permanent_uuid: "85aafda77b3c49a59d9ddd28b7cd9709" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36143 } } peers { permanent_uuid: "279549c47ec9448e8d9698eedf3d88c0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39691 } }
I20260811 11:59:27.042737 27714 ts_tablet_manager.cc:1428] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 11:59:27.043474 27716 ts_tablet_manager.cc:1428] T 52b0317878394561b929a5c31ad340cc P 279549c47ec9448e8d9698eedf3d88c0: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 11:59:27.043474 27715 ts_tablet_manager.cc:1428] T 52b0317878394561b929a5c31ad340cc P 85aafda77b3c49a59d9ddd28b7cd9709: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 11:59:27.043804 27406 heartbeater.cc:503] Master 127.6.143.190:41639 was elected leader, sending a full tablet report...
I20260811 11:59:27.043865 27694 heartbeater.cc:503] Master 127.6.143.190:41639 was elected leader, sending a full tablet report...
W20260811 11:59:27.081449 27407 tablet.cc:2369] T 52b0317878394561b929a5c31ad340cc P 279549c47ec9448e8d9698eedf3d88c0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:59:27.103235 27722 raft_consensus.cc:482] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:59:27.103489 27722 raft_consensus.cc:504] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16eeeadf08c54649ad1271be06ce250a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41671 } } peers { permanent_uuid: "85aafda77b3c49a59d9ddd28b7cd9709" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36143 } } peers { permanent_uuid: "279549c47ec9448e8d9698eedf3d88c0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39691 } }
I20260811 11:59:27.104668 27722 leader_election.cc:288] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 85aafda77b3c49a59d9ddd28b7cd9709 (127.6.143.131:36143), 279549c47ec9448e8d9698eedf3d88c0 (127.6.143.129:39691)
I20260811 11:59:27.108947 27709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:27.105038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:44217 (local address 127.6.143.131:36143)
0811 11:59:27.105178 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:27.105185 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:27.105203 (+ 18us) server_negotiation.cc:407] Connection header received
0811 11:59:27.105379 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:27.105386 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:27.105445 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:27.105536 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:27.105832 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:27.106588 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:27.107811 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:27.108439 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:27.108528 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:27.108569 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:27.108792 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 11:59:27.108847 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 11:59:27.108968 27555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:27.104956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:36143 (local address 127.6.143.130:44217)
0811 11:59:27.105128 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:27.105145 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:27.105247 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:27.105550 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:27.105553 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:27.105578 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:27.105679 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:27.105684 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:27.106684 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:27.106688 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:27.107676 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:27.107686 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:27.108544 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:27.108547 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:27.108648 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:27.108673 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 11:59:27.108754 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 11:59:27.108822 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 11:59:27.109513 27711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:27.105375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:51503 (local address 127.6.143.129:39691)
0811 11:59:27.105510 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:27.105514 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:27.105850 (+ 336us) server_negotiation.cc:407] Connection header received
0811 11:59:27.105957 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:27.105959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:27.106002 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:27.106063 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:27.106358 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:27.107086 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:27.108379 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:27.108923 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:27.109044 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:27.109095 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:27.109357 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 11:59:27.109427 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 11:59:27.109663 27644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52b0317878394561b929a5c31ad340cc" candidate_uuid: "16eeeadf08c54649ad1271be06ce250a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85aafda77b3c49a59d9ddd28b7cd9709" is_pre_election: true
I20260811 11:59:27.109899 27725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:27.105259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:39691 (local address 127.6.143.130:51503)
0811 11:59:27.105698 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:27.105709 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:27.105860 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:27.106103 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:27.106106 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:27.106124 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:27.106253 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:27.106259 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:27.107212 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:27.107217 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:27.108260 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:27.108271 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:27.109080 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:27.109084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:27.109191 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:27.109216 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 11:59:27.109675 (+ 459us) client_negotiation.cc:240] Negotiation successful
0811 11:59:27.109741 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":119,"threads_started":1}
I20260811 11:59:27.110013 27644 raft_consensus.cc:2454] T 52b0317878394561b929a5c31ad340cc P 85aafda77b3c49a59d9ddd28b7cd9709 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16eeeadf08c54649ad1271be06ce250a in term 0.
I20260811 11:59:27.110496 27429 leader_election.cc:302] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16eeeadf08c54649ad1271be06ce250a, 85aafda77b3c49a59d9ddd28b7cd9709; no voters:
I20260811 11:59:27.110518 27354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52b0317878394561b929a5c31ad340cc" candidate_uuid: "16eeeadf08c54649ad1271be06ce250a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "279549c47ec9448e8d9698eedf3d88c0" is_pre_election: true
I20260811 11:59:27.110773 27722 raft_consensus.cc:2781] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:59:27.110809 27354 raft_consensus.cc:2454] T 52b0317878394561b929a5c31ad340cc P 279549c47ec9448e8d9698eedf3d88c0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 16eeeadf08c54649ad1271be06ce250a in term 0.
I20260811 11:59:27.110880 27722 raft_consensus.cc:482] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:59:27.110935 27722 raft_consensus.cc:3037] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:27.112185 27722 raft_consensus.cc:504] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16eeeadf08c54649ad1271be06ce250a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41671 } } peers { permanent_uuid: "85aafda77b3c49a59d9ddd28b7cd9709" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36143 } } peers { permanent_uuid: "279549c47ec9448e8d9698eedf3d88c0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39691 } }
I20260811 11:59:27.112509 27722 leader_election.cc:288] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a [CANDIDATE]: Term 1 election: Requested vote from peers 85aafda77b3c49a59d9ddd28b7cd9709 (127.6.143.131:36143), 279549c47ec9448e8d9698eedf3d88c0 (127.6.143.129:39691)
I20260811 11:59:27.112850 27644 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52b0317878394561b929a5c31ad340cc" candidate_uuid: "16eeeadf08c54649ad1271be06ce250a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85aafda77b3c49a59d9ddd28b7cd9709"
I20260811 11:59:27.112864 27354 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52b0317878394561b929a5c31ad340cc" candidate_uuid: "16eeeadf08c54649ad1271be06ce250a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "279549c47ec9448e8d9698eedf3d88c0"
I20260811 11:59:27.112973 27354 raft_consensus.cc:3037] T 52b0317878394561b929a5c31ad340cc P 279549c47ec9448e8d9698eedf3d88c0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:27.112973 27644 raft_consensus.cc:3037] T 52b0317878394561b929a5c31ad340cc P 85aafda77b3c49a59d9ddd28b7cd9709 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:27.114256 27354 raft_consensus.cc:2454] T 52b0317878394561b929a5c31ad340cc P 279549c47ec9448e8d9698eedf3d88c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16eeeadf08c54649ad1271be06ce250a in term 1.
I20260811 11:59:27.114265 27644 raft_consensus.cc:2454] T 52b0317878394561b929a5c31ad340cc P 85aafda77b3c49a59d9ddd28b7cd9709 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 16eeeadf08c54649ad1271be06ce250a in term 1.
I20260811 11:59:27.114563 27429 leader_election.cc:302] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 16eeeadf08c54649ad1271be06ce250a, 85aafda77b3c49a59d9ddd28b7cd9709; no voters:
I20260811 11:59:27.114718 27722 raft_consensus.cc:2781] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:27.114940 27722 raft_consensus.cc:686] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a [term 1 LEADER]: Becoming Leader. State: Replica: 16eeeadf08c54649ad1271be06ce250a, State: Running, Role: LEADER
I20260811 11:59:27.115267 27722 consensus_queue.cc:237] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16eeeadf08c54649ad1271be06ce250a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41671 } } peers { permanent_uuid: "85aafda77b3c49a59d9ddd28b7cd9709" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36143 } } peers { permanent_uuid: "279549c47ec9448e8d9698eedf3d88c0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39691 } }
I20260811 11:59:27.118394 27204 catalog_manager.cc:5495] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a reported cstate change: term changed from 0 to 1, leader changed from <none> to 16eeeadf08c54649ad1271be06ce250a (127.6.143.130). New cstate: current_term: 1 leader_uuid: "16eeeadf08c54649ad1271be06ce250a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "16eeeadf08c54649ad1271be06ce250a" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 41671 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "85aafda77b3c49a59d9ddd28b7cd9709" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36143 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "279549c47ec9448e8d9698eedf3d88c0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39691 } health_report { overall_health: UNKNOWN } } }
I20260811 11:59:27.143191 27707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:27.139966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58524 (local address 127.6.143.130:41671)
0811 11:59:27.140079 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:27.140083 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:27.140097 (+ 14us) server_negotiation.cc:407] Connection header received
0811 11:59:27.140215 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:27.140217 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:27.140268 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:27.140353 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:59:27.140567 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:27.141042 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:27.141862 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:27.142065 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:27.142141 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:27.142432 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:59:27.142802 (+ 370us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:59:27.142920 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:27.143065 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 11:59:27.143095 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 11:59:27.149358 27644 raft_consensus.cc:1261] T 52b0317878394561b929a5c31ad340cc P 85aafda77b3c49a59d9ddd28b7cd9709 [term 1 FOLLOWER]: Refusing update from remote peer 16eeeadf08c54649ad1271be06ce250a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:59:27.149358 27354 raft_consensus.cc:1261] T 52b0317878394561b929a5c31ad340cc P 279549c47ec9448e8d9698eedf3d88c0 [term 1 FOLLOWER]: Refusing update from remote peer 16eeeadf08c54649ad1271be06ce250a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:59:27.150135 27722 consensus_queue.cc:1035] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a [LEADER]: Connected to new peer: Peer: permanent_uuid: "279549c47ec9448e8d9698eedf3d88c0" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 39691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:59:27.150342 27726 consensus_queue.cc:1035] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a [LEADER]: Connected to new peer: Peer: permanent_uuid: "85aafda77b3c49a59d9ddd28b7cd9709" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 36143 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:59:27.159145 27730 mvcc.cc:205] Tried to move back new op lower bound from 7317297427035881472 to 7317297426907070464. Current Snapshot: MvccSnapshot[applied={T|T < 7317297427035881472}]
I20260811 11:59:27.161777 27731 mvcc.cc:205] Tried to move back new op lower bound from 7317297427035881472 to 7317297426907070464. Current Snapshot: MvccSnapshot[applied={T|T < 7317297427035881472 or (T in {7317297427035881472})}]
I20260811 11:59:27.166566 27203 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:59:27.166764 27203 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60860:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:59:27.169230 27203 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3347eef1c6cd41f0b5519fa58753fbf0: Sending DeleteTablet for 3 replicas of tablet 52b0317878394561b929a5c31ad340cc
I20260811 11:59:27.169958 27480 tablet_service.cc:1514] Processing DeleteTablet for tablet 52b0317878394561b929a5c31ad340cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:59:27 UTC) from {username='slave'} at 127.0.0.1:58520
I20260811 11:59:27.170192 27332 tablet_service.cc:1514] Processing DeleteTablet for tablet 52b0317878394561b929a5c31ad340cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:59:27 UTC) from {username='slave'} at 127.0.0.1:46450
I20260811 11:59:27.170198 27622 tablet_service.cc:1514] Processing DeleteTablet for tablet 52b0317878394561b929a5c31ad340cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:59:27 UTC) from {username='slave'} at 127.0.0.1:51494
I20260811 11:59:27.170754 27738 tablet_replica.cc:330] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a: stopping tablet replica
I20260811 11:59:27.171128 27738 raft_consensus.cc:2229] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:59:27.171162 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 27269
I20260811 11:59:27.172609 27738 raft_consensus.cc:2258] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:59:27.174765 27738 ts_tablet_manager.cc:1905] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:59:27.178454 27738 ts_tablet_manager.cc:1918] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:59:27.178584 27738 log.cc:1196] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-1/wal/wals/52b0317878394561b929a5c31ad340cc
I20260811 11:59:27.178886 27738 ts_tablet_manager.cc:1939] T 52b0317878394561b929a5c31ad340cc P 16eeeadf08c54649ad1271be06ce250a: Deleting consensus metadata
I20260811 11:59:27.179991 27189 catalog_manager.cc:4841] TS 16eeeadf08c54649ad1271be06ce250a (127.6.143.130:41671): tablet 52b0317878394561b929a5c31ad340cc (table test-table [id=ca8a0d42ccd141f9b2ac88f8f005c12c]) successfully deleted
I20260811 11:59:27.180037 27740 tablet_replica.cc:330] T 52b0317878394561b929a5c31ad340cc P 85aafda77b3c49a59d9ddd28b7cd9709: stopping tablet replica
I20260811 11:59:27.180377 27740 raft_consensus.cc:2229] T 52b0317878394561b929a5c31ad340cc P 85aafda77b3c49a59d9ddd28b7cd9709 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:59:27.180758 27740 raft_consensus.cc:2258] T 52b0317878394561b929a5c31ad340cc P 85aafda77b3c49a59d9ddd28b7cd9709 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:59:27.182279 27740 ts_tablet_manager.cc:1905] T 52b0317878394561b929a5c31ad340cc P 85aafda77b3c49a59d9ddd28b7cd9709: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:59:27.185799 27740 ts_tablet_manager.cc:1918] T 52b0317878394561b929a5c31ad340cc P 85aafda77b3c49a59d9ddd28b7cd9709: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:59:27.185905 27740 log.cc:1196] T 52b0317878394561b929a5c31ad340cc P 85aafda77b3c49a59d9ddd28b7cd9709: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786449319379472-6718-0/minicluster-data/ts-2/wal/wals/52b0317878394561b929a5c31ad340cc
I20260811 11:59:27.186239 27740 ts_tablet_manager.cc:1939] T 52b0317878394561b929a5c31ad340cc P 85aafda77b3c49a59d9ddd28b7cd9709: Deleting consensus metadata
I20260811 11:59:27.187577 27189 catalog_manager.cc:4841] TS 85aafda77b3c49a59d9ddd28b7cd9709 (127.6.143.131:36143): tablet 52b0317878394561b929a5c31ad340cc (table test-table [id=ca8a0d42ccd141f9b2ac88f8f005c12c]) successfully deleted
W20260811 11:59:27.187866 27189 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.143.129:39691 (error 108)
I20260811 11:59:27.188051 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 27409
I20260811 11:59:27.190601 27710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:27.190215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:39691 (local address 127.0.0.1:46462)
0811 11:59:27.190461 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:27.190529 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.129:39691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
W20260811 11:59:27.190765 27189 catalog_manager.cc:4584] TS 279549c47ec9448e8d9698eedf3d88c0 (127.6.143.129:39691): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 52b0317878394561b929a5c31ad340cc: Network error: Client connection negotiation failed: client connection to 127.6.143.129:39691: connect: Connection refused (error 111)
I20260811 11:59:27.197904 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 27556
I20260811 11:59:27.205476 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 27170
2026-08-11T11:59:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1106 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:59:27 dist-test-slave-zkpd krb5kdc[27744](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:59:27 dist-test-slave-zkpd krb5kdc[27744](info): set up 2 sockets
Aug 11 11:59:27 dist-test-slave-zkpd krb5kdc[27744](info): commencing operation
krb5kdc: starting...
W20260811 11:59:28.289765 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:59:28 dist-test-slave-zkpd krb5kdc[27744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:59:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:59:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
I20260811 11:59:28.416672 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:41885
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:42189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:41885
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:28.528250 27757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:28.528522 27757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:28.528595 27757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:28.532536 27757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:59:28.532614 27757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:28.532655 27757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:28.532689 27757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:59:28.532722 27757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:59:28.537766 27757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:42189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:41885
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:41885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:28.538733 27757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:28.539700 27757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:28.547978 27766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:28.548069 27757 server_base.cc:1034] running on GCE node
W20260811 11:59:28.548012 27762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:28.547986 27763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:28.548918 27757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:28.549937 27757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:28.551102 27757 hybrid_clock.cc:648] HybridClock initialized: now 1786449568551087 us; error 25 us; skew 500 ppm
Aug 11 11:59:28 dist-test-slave-zkpd krb5kdc[27744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449568, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:28.553925 27757 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:59:28.554776 27757 webserver.cc:460] Webserver started at http://127.6.143.190:41223/ using document root <none> and password file <none>
I20260811 11:59:28.555290 27757 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:28.555349 27757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:28.555519 27757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:28.557534 27757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "c12debeef2bb4384add8f6bd160fb53c"
format_stamp: "Formatted at 2026-08-11 11:59:28 on dist-test-slave-zkpd"
server_key: "50f7cf32245623f1e833c9ba43317a0d"
server_key_iv: "7c90e7dd3a3fb2f367217c09832c5386"
server_key_version: "encryptionkey@0"
I20260811 11:59:28.558050 27757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "c12debeef2bb4384add8f6bd160fb53c"
format_stamp: "Formatted at 2026-08-11 11:59:28 on dist-test-slave-zkpd"
server_key: "50f7cf32245623f1e833c9ba43317a0d"
server_key_iv: "7c90e7dd3a3fb2f367217c09832c5386"
server_key_version: "encryptionkey@0"
I20260811 11:59:28.561661 27757 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260811 11:59:28.564038 27774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:28.565277 27757 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260811 11:59:28.565428 27757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "c12debeef2bb4384add8f6bd160fb53c"
format_stamp: "Formatted at 2026-08-11 11:59:28 on dist-test-slave-zkpd"
server_key: "50f7cf32245623f1e833c9ba43317a0d"
server_key_iv: "7c90e7dd3a3fb2f367217c09832c5386"
server_key_version: "encryptionkey@0"
I20260811 11:59:28.565541 27757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:28.596939 27757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:28.599680 27757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:28.599903 27757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:28.607137 27757 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:41885
I20260811 11:59:28.607164 27832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:41885 every 8 connection(s)
I20260811 11:59:28.608045 27757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:59:28.610889 27833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:28.613198 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 27757
I20260811 11:59:28.613350 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:59:28.613659 6718 external_mini_cluster.cc:1428] Setting key 7adde5180e7c09dbc219e390691b5027
I20260811 11:59:28.617213 27833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c12debeef2bb4384add8f6bd160fb53c: Bootstrap starting.
I20260811 11:59:28.619830 27833 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c12debeef2bb4384add8f6bd160fb53c: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:28.620754 27833 log.cc:824] T 00000000000000000000000000000000 P c12debeef2bb4384add8f6bd160fb53c: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:28.622800 27833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c12debeef2bb4384add8f6bd160fb53c: No bootstrap required, opened a new log
Aug 11 11:59:28 dist-test-slave-zkpd krb5kdc[27744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:59:28.625947 27833 raft_consensus.cc:348] T 00000000000000000000000000000000 P c12debeef2bb4384add8f6bd160fb53c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c12debeef2bb4384add8f6bd160fb53c" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 41885 } }
I20260811 11:59:28.626157 27833 raft_consensus.cc:374] T 00000000000000000000000000000000 P c12debeef2bb4384add8f6bd160fb53c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:28.626226 27833 raft_consensus.cc:729] T 00000000000000000000000000000000 P c12debeef2bb4384add8f6bd160fb53c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c12debeef2bb4384add8f6bd160fb53c, State: Initialized, Role: FOLLOWER
I20260811 11:59:28.626718 27833 consensus_queue.cc:260] T 00000000000000000000000000000000 P c12debeef2bb4384add8f6bd160fb53c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c12debeef2bb4384add8f6bd160fb53c" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 41885 } }
I20260811 11:59:28.626896 27833 raft_consensus.cc:388] T 00000000000000000000000000000000 P c12debeef2bb4384add8f6bd160fb53c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:59:28.626976 27833 raft_consensus.cc:482] T 00000000000000000000000000000000 P c12debeef2bb4384add8f6bd160fb53c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:59:28.627123 27833 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c12debeef2bb4384add8f6bd160fb53c [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:28.628268 27833 raft_consensus.cc:504] T 00000000000000000000000000000000 P c12debeef2bb4384add8f6bd160fb53c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c12debeef2bb4384add8f6bd160fb53c" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 41885 } }
I20260811 11:59:28.628645 27833 leader_election.cc:302] T 00000000000000000000000000000000 P c12debeef2bb4384add8f6bd160fb53c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c12debeef2bb4384add8f6bd160fb53c; no voters:
I20260811 11:59:28.628995 27833 leader_election.cc:288] T 00000000000000000000000000000000 P c12debeef2bb4384add8f6bd160fb53c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:59:28.629104 27838 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c12debeef2bb4384add8f6bd160fb53c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:28.629317 27838 raft_consensus.cc:686] T 00000000000000000000000000000000 P c12debeef2bb4384add8f6bd160fb53c [term 1 LEADER]: Becoming Leader. State: Replica: c12debeef2bb4384add8f6bd160fb53c, State: Running, Role: LEADER
I20260811 11:59:28.629628 27838 consensus_queue.cc:237] T 00000000000000000000000000000000 P c12debeef2bb4384add8f6bd160fb53c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c12debeef2bb4384add8f6bd160fb53c" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 41885 } }
I20260811 11:59:28.629895 27836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:28.615081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42732 (local address 127.6.143.190:41885)
0811 11:59:28.615520 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:28.615534 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:28.615568 (+ 34us) server_negotiation.cc:407] Connection header received
0811 11:59:28.616361 (+ 793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:28.616381 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:28.616700 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:28.617047 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:28.617828 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:28.619213 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:28.620487 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:28.621037 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:28.621233 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:28.624058 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:28.624063 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:28.624076 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:28.624103 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:28.626484 (+ 2381us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:28.627025 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:28.627031 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:28.627037 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:28.627182 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:28.627528 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:28.627530 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:28.627531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:28.627994 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:28.628165 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:28.628533 (+ 368us) server_negotiation.cc:299] Negotiation successful
0811 11:59:28.628881 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":118,"threads_started":1}
I20260811 11:59:28.630331 27833 sys_catalog.cc:564] T 00000000000000000000000000000000 P c12debeef2bb4384add8f6bd160fb53c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:59:28.630851 27838 sys_catalog.cc:455] T 00000000000000000000000000000000 P c12debeef2bb4384add8f6bd160fb53c [sys.catalog]: SysCatalogTable state changed. Reason: New leader c12debeef2bb4384add8f6bd160fb53c. Latest consensus state: current_term: 1 leader_uuid: "c12debeef2bb4384add8f6bd160fb53c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c12debeef2bb4384add8f6bd160fb53c" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 41885 } } }
I20260811 11:59:28.630959 27838 sys_catalog.cc:458] T 00000000000000000000000000000000 P c12debeef2bb4384add8f6bd160fb53c [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:28.631338 27839 sys_catalog.cc:455] T 00000000000000000000000000000000 P c12debeef2bb4384add8f6bd160fb53c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c12debeef2bb4384add8f6bd160fb53c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c12debeef2bb4384add8f6bd160fb53c" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 41885 } } }
I20260811 11:59:28.631445 27839 sys_catalog.cc:458] T 00000000000000000000000000000000 P c12debeef2bb4384add8f6bd160fb53c [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:28.632450 27842 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:59:28.635221 27842 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:59:28.641256 27842 catalog_manager.cc:1347] Generated new cluster ID: 5760c084635f4d1daf4c732ee5cd55a0
I20260811 11:59:28.641345 27842 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:59:28.654099 27842 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:59:28.655045 27842 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:59:28.659916 27842 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c12debeef2bb4384add8f6bd160fb53c: Generated new TSK 0
I20260811 11:59:28.660621 27842 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.143.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
Entry for principal kudu/127.6.143.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab.
I20260811 11:59:28.690321 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:0
--local_ip_for_outbound_sockets=127.6.143.129
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:41885
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:42189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:28.802165 27861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:28.802400 27861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:28.802443 27861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:28.806383 27861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:28.806432 27861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:28.806486 27861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:59:28.811614 27861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:42189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.129.keytab
--principal=kudu/127.6.143.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:41885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:28.812497 27861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:28.813402 27861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:28.822762 27867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:28.822904 27861 server_base.cc:1034] running on GCE node
W20260811 11:59:28.822808 27868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:28.822763 27870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:28.823474 27861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:28.824097 27861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:28.825268 27861 hybrid_clock.cc:648] HybridClock initialized: now 1786449568825253 us; error 25 us; skew 500 ppm
Aug 11 11:59:28 dist-test-slave-zkpd krb5kdc[27744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449568, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:28.828166 27861 init.cc:377] Logged in from keytab as kudu/127.6.143.129@KRBTEST.COM (short username kudu)
I20260811 11:59:28.828954 27861 webserver.cc:460] Webserver started at http://127.6.143.129:42223/ using document root <none> and password file <none>
I20260811 11:59:28.829519 27861 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:28.829576 27861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:28.829789 27861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:28.832001 27861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-0/data/instance:
uuid: "bbfd5fded9194fe0a91aee4b56f04398"
format_stamp: "Formatted at 2026-08-11 11:59:28 on dist-test-slave-zkpd"
server_key: "b83c49075147e20cb268f1ed5dbda86f"
server_key_iv: "856a406c09a52995a3d068fc78f94b80"
server_key_version: "encryptionkey@0"
I20260811 11:59:28.832592 27861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance:
uuid: "bbfd5fded9194fe0a91aee4b56f04398"
format_stamp: "Formatted at 2026-08-11 11:59:28 on dist-test-slave-zkpd"
server_key: "b83c49075147e20cb268f1ed5dbda86f"
server_key_iv: "856a406c09a52995a3d068fc78f94b80"
server_key_version: "encryptionkey@0"
I20260811 11:59:28.836220 27861 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 11:59:28.838546 27877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:28.839723 27861 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:59:28.839857 27861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "bbfd5fded9194fe0a91aee4b56f04398"
format_stamp: "Formatted at 2026-08-11 11:59:28 on dist-test-slave-zkpd"
server_key: "b83c49075147e20cb268f1ed5dbda86f"
server_key_iv: "856a406c09a52995a3d068fc78f94b80"
server_key_version: "encryptionkey@0"
I20260811 11:59:28.840020 27861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:28.878592 27861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:28.881935 27861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:28.882139 27861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:28.882746 27861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:59:28.883576 27861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:59:28.883647 27861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:28.883719 27861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:59:28.883771 27861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:28.893029 27861 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:42577
I20260811 11:59:28.893065 27999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:42577 every 8 connection(s)
I20260811 11:59:28.893957 27861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:59:28.897032 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 27861
I20260811 11:59:28.897187 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance
I20260811 11:59:28.897650 6718 external_mini_cluster.cc:1428] Setting key 9216632d7b6dc8269842dbc777978245
Aug 11 11:59:28 dist-test-slave-zkpd krb5kdc[27744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449568, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.129@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.143.130@KRBTEST.COM; defaulting to no policy
I20260811 11:59:28.909811 27836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:28.895819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:46453 (local address 127.6.143.190:41885)
0811 11:59:28.896033 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:28.896038 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:28.896877 (+ 839us) server_negotiation.cc:407] Connection header received
0811 11:59:28.898717 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:28.898721 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:28.898765 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:28.898856 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:28.900614 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:28.901134 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:28.902497 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:28.902879 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:28.902976 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:28.906139 (+ 3163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:28.906142 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:28.906144 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:28.906169 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:28.907636 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:28.908250 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:28.908254 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:28.908256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:28.908340 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:28.908708 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:28.908712 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:28.908713 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:28.908892 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:28.909015 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:28.909520 (+ 505us) server_negotiation.cc:299] Negotiation successful
0811 11:59:28.909646 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260811 11:59:28.910630 28002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:28.896196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:41885 (local address 127.6.143.129:46453)
0811 11:59:28.896659 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:28.896729 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:28.898495 (+ 1766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:28.899078 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:28.899087 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:28.899638 (+ 551us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:59:28.900443 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:28.900469 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:28.901271 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:28.901279 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:28.902358 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:28.902369 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:28.903162 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:28.903170 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:28.903425 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:28.904151 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:59:28.904179 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:59:28.905969 (+ 1790us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:59:28.907778 (+ 1809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:28.907783 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:28.907799 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:28.908137 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:28.908453 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:28.908456 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:28.908458 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:28.908600 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:28.909015 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:59:28.909022 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:59:28.909278 (+ 256us) client_negotiation.cc:769] Sending connection context
0811 11:59:28.909458 (+ 180us) client_negotiation.cc:240] Negotiation successful
0811 11:59:28.909769 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":124,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.130@KRBTEST.COM" created.
I20260811 11:59:28.912133 28000 heartbeater.cc:347] Connected to a master server at 127.6.143.190:41885
I20260811 11:59:28.912405 28000 heartbeater.cc:464] Registering TS with master...
I20260811 11:59:28.912895 28000 heartbeater.cc:511] Master 127.6.143.190:41885 requested a full tablet report, sending...
I20260811 11:59:28.914518 27794 ts_manager.cc:194] Registered new tserver with Master: bbfd5fded9194fe0a91aee4b56f04398 (127.6.143.129:42577)
I20260811 11:59:28.915638 27794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.129@KRBTEST.COM'} at 127.6.143.129:46453
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
Entry for principal kudu/127.6.143.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab.
I20260811 11:59:28.927145 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:0
--local_ip_for_outbound_sockets=127.6.143.130
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:41885
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:42189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:29.038528 28005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:29.038767 28005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:29.038807 28005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:29.042675 28005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:29.042721 28005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:29.042793 28005 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:59:29.047765 28005 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:42189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.130.keytab
--principal=kudu/127.6.143.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:41885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28005
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:29.048735 28005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:29.049656 28005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:29.059149 28014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:29.059206 28012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:29.059154 28011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:29.059366 28005 server_base.cc:1034] running on GCE node
I20260811 11:59:29.059823 28005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:29.060505 28005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:29.061679 28005 hybrid_clock.cc:648] HybridClock initialized: now 1786449569061655 us; error 28 us; skew 500 ppm
Aug 11 11:59:29 dist-test-slave-zkpd krb5kdc[27744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449569, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:29.064559 28005 init.cc:377] Logged in from keytab as kudu/127.6.143.130@KRBTEST.COM (short username kudu)
I20260811 11:59:29.065415 28005 webserver.cc:460] Webserver started at http://127.6.143.130:42485/ using document root <none> and password file <none>
I20260811 11:59:29.065958 28005 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:29.066030 28005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:29.066320 28005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:29.068486 28005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-1/data/instance:
uuid: "3e36d8c5a02a4b72b06934a7e4b21fd0"
format_stamp: "Formatted at 2026-08-11 11:59:29 on dist-test-slave-zkpd"
server_key: "82852ff1a83df2bd69d25b6837dcb1a3"
server_key_iv: "b9e6c20117d83dd5d0f73a84c4ff9d8a"
server_key_version: "encryptionkey@0"
I20260811 11:59:29.069124 28005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance:
uuid: "3e36d8c5a02a4b72b06934a7e4b21fd0"
format_stamp: "Formatted at 2026-08-11 11:59:29 on dist-test-slave-zkpd"
server_key: "82852ff1a83df2bd69d25b6837dcb1a3"
server_key_iv: "b9e6c20117d83dd5d0f73a84c4ff9d8a"
server_key_version: "encryptionkey@0"
I20260811 11:59:29.073022 28005 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 11:59:29.075335 28021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:29.076578 28005 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:59:29.076714 28005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "3e36d8c5a02a4b72b06934a7e4b21fd0"
format_stamp: "Formatted at 2026-08-11 11:59:29 on dist-test-slave-zkpd"
server_key: "82852ff1a83df2bd69d25b6837dcb1a3"
server_key_iv: "b9e6c20117d83dd5d0f73a84c4ff9d8a"
server_key_version: "encryptionkey@0"
I20260811 11:59:29.076819 28005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:29.088043 28005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:29.090620 28005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:29.090837 28005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:29.091360 28005 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:59:29.092219 28005 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:59:29.092288 28005 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:29.092360 28005 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:59:29.092412 28005 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:29.100976 28005 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:35297
I20260811 11:59:29.101006 28146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:35297 every 8 connection(s)
I20260811 11:59:29.101835 28005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
I20260811 11:59:29.103410 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 28005
I20260811 11:59:29.103582 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance
I20260811 11:59:29.104009 6718 external_mini_cluster.cc:1428] Setting key a8af05db8217d89743f871421df69b89
Aug 11 11:59:29 dist-test-slave-zkpd krb5kdc[27744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449569, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.130@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.143.131@KRBTEST.COM; defaulting to no policy
I20260811 11:59:29.116312 27836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:29.103702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:36055 (local address 127.6.143.190:41885)
0811 11:59:29.103852 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:29.103857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:29.104920 (+ 1063us) server_negotiation.cc:407] Connection header received
0811 11:59:29.106326 (+ 1406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:29.106330 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:29.106382 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:29.106461 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:29.107585 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.108201 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.109304 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.109522 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.109632 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:29.112433 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:29.112436 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:29.112438 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:29.112461 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:29.114126 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:29.114628 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:29.114632 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:29.114634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:29.114726 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:29.115167 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:29.115172 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:29.115174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:29.115358 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:29.115470 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:29.116028 (+ 558us) server_negotiation.cc:299] Negotiation successful
0811 11:59:29.116153 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.131@KRBTEST.COM" created.
I20260811 11:59:29.117169 28149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:29.104209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:41885 (local address 127.6.143.130:36055)
0811 11:59:29.104754 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:29.104813 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:29.106116 (+ 1303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:29.106632 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:29.106638 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:29.106993 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:59:29.107444 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:29.107457 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.108339 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.108343 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:29.109170 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:29.109187 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.109668 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.109673 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:29.109844 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:29.110448 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:59:29.110463 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:59:29.112245 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:59:29.114243 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:29.114249 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:29.114264 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:29.114517 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:29.114821 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:29.114825 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:29.114827 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:29.115057 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:29.115470 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:59:29.115478 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:59:29.115719 (+ 241us) client_negotiation.cc:769] Sending connection context
0811 11:59:29.115976 (+ 257us) client_negotiation.cc:240] Negotiation successful
0811 11:59:29.116329 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":151,"threads_started":1}
I20260811 11:59:29.119215 28147 heartbeater.cc:347] Connected to a master server at 127.6.143.190:41885
I20260811 11:59:29.119462 28147 heartbeater.cc:464] Registering TS with master...
I20260811 11:59:29.120010 28147 heartbeater.cc:511] Master 127.6.143.190:41885 requested a full tablet report, sending...
I20260811 11:59:29.121233 27794 ts_manager.cc:194] Registered new tserver with Master: 3e36d8c5a02a4b72b06934a7e4b21fd0 (127.6.143.130:35297)
I20260811 11:59:29.122126 27794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.130@KRBTEST.COM'} at 127.6.143.130:36055
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
Entry for principal kudu/127.6.143.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab.
I20260811 11:59:29.132890 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:0
--local_ip_for_outbound_sockets=127.6.143.131
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:41885
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:42189
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:29.246801 28152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:29.247040 28152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:29.247079 28152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:29.250924 28152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:29.250974 28152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:29.251029 28152 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:59:29.255968 28152 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:42189
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.131.keytab
--principal=kudu/127.6.143.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:41885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:29.256842 28152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:29.257745 28152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:29.267024 28157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:29.267024 28160 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:29.267037 28158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:29.267247 28152 server_base.cc:1034] running on GCE node
I20260811 11:59:29.267772 28152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:29.268548 28152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:29.269731 28152 hybrid_clock.cc:648] HybridClock initialized: now 1786449569269712 us; error 33 us; skew 500 ppm
Aug 11 11:59:29 dist-test-slave-zkpd krb5kdc[27744](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449569, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:29.272920 28152 init.cc:377] Logged in from keytab as kudu/127.6.143.131@KRBTEST.COM (short username kudu)
I20260811 11:59:29.274179 28152 webserver.cc:460] Webserver started at http://127.6.143.131:34785/ using document root <none> and password file <none>
I20260811 11:59:29.274724 28152 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:29.274781 28152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:29.274942 28152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:29.277033 28152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-2/data/instance:
uuid: "2e6a6dab78334d86b83da60323006260"
format_stamp: "Formatted at 2026-08-11 11:59:29 on dist-test-slave-zkpd"
server_key: "47ac24082d4b6102826f9b606da35a81"
server_key_iv: "0638e06eee65de3425dab3886b67f929"
server_key_version: "encryptionkey@0"
I20260811 11:59:29.277556 28152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance:
uuid: "2e6a6dab78334d86b83da60323006260"
format_stamp: "Formatted at 2026-08-11 11:59:29 on dist-test-slave-zkpd"
server_key: "47ac24082d4b6102826f9b606da35a81"
server_key_iv: "0638e06eee65de3425dab3886b67f929"
server_key_version: "encryptionkey@0"
I20260811 11:59:29.281402 28152 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 11:59:29.283802 28167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:29.285046 28152 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:59:29.285144 28152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "2e6a6dab78334d86b83da60323006260"
format_stamp: "Formatted at 2026-08-11 11:59:29 on dist-test-slave-zkpd"
server_key: "47ac24082d4b6102826f9b606da35a81"
server_key_iv: "0638e06eee65de3425dab3886b67f929"
server_key_version: "encryptionkey@0"
I20260811 11:59:29.285231 28152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:29.301446 28152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:29.304500 28152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:29.304654 28152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:29.305197 28152 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:59:29.305984 28152 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:59:29.306038 28152 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:29.306078 28152 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:59:29.306093 28152 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:29.315187 28152 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:40065
I20260811 11:59:29.315223 28290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:40065 every 8 connection(s)
I20260811 11:59:29.316110 28152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:59:29.319250 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 28152
I20260811 11:59:29.319418 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance
I20260811 11:59:29.319718 6718 external_mini_cluster.cc:1428] Setting key 6d860e2207614b28a845b14a478970ab
Aug 11 11:59:29 dist-test-slave-zkpd krb5kdc[27744](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449569, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.131@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:59:29.330155 27836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:29.317922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:47811 (local address 127.6.143.190:41885)
0811 11:59:29.318068 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:29.318072 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:29.318943 (+ 871us) server_negotiation.cc:407] Connection header received
0811 11:59:29.320538 (+ 1595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:29.320542 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:29.320600 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:29.320693 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:29.322198 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.322734 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.323828 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.324057 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.324142 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:29.326460 (+ 2318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:29.326464 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:29.326467 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:29.326499 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:29.328086 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:29.328596 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:29.328600 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:29.328602 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:29.328686 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:29.329093 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:29.329097 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:29.329099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:29.329291 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:29.329404 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:29.329934 (+ 530us) server_negotiation.cc:299] Negotiation successful
0811 11:59:29.330036 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 11:59:29.331022 28294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:29.318281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:41885 (local address 127.6.143.131:47811)
0811 11:59:29.318779 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:29.318822 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:29.320327 (+ 1505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:29.320922 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:29.320934 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:29.321396 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:59:29.322026 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:29.322043 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.322853 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.322855 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:29.323702 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:29.323711 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.324186 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.324189 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:29.324323 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:29.324820 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:59:29.324836 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:59:29.326293 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:59:29.328209 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:29.328215 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:29.328229 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:29.328486 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:29.328784 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:29.328788 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:29.328790 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:29.328942 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:29.329401 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:59:29.329409 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:59:29.329641 (+ 232us) client_negotiation.cc:769] Sending connection context
0811 11:59:29.329890 (+ 249us) client_negotiation.cc:240] Negotiation successful
0811 11:59:29.330142 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":119,"threads_started":1}
I20260811 11:59:29.332226 28291 heartbeater.cc:347] Connected to a master server at 127.6.143.190:41885
I20260811 11:59:29.332479 28291 heartbeater.cc:464] Registering TS with master...
I20260811 11:59:29.333132 28291 heartbeater.cc:511] Master 127.6.143.190:41885 requested a full tablet report, sending...
I20260811 11:59:29.334250 27794 ts_manager.cc:194] Registered new tserver with Master: 2e6a6dab78334d86b83da60323006260 (127.6.143.131:40065)
I20260811 11:59:29.334815 27794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.143.131@KRBTEST.COM'} at 127.6.143.131:47811
I20260811 11:59:29.345145 6718 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:59:29.354734 27836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:29.347065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42740 (local address 127.6.143.190:41885)
0811 11:59:29.347167 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:29.347170 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:29.347413 (+ 243us) server_negotiation.cc:407] Connection header received
0811 11:59:29.347503 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:29.347506 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:29.347547 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:29.347618 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:29.348360 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.348968 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.349795 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.350024 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.350113 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:29.351401 (+ 1288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:29.351404 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:29.351406 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:29.351450 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:29.353124 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:29.353527 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:29.353530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:29.353531 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:29.353613 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:29.353886 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:29.353889 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:29.353890 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:29.354105 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:29.354225 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:29.354394 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 11:59:29.354484 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 11:59:29.357564 27794 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42740:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:59:29.360657 27794 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:59:29.375967 28309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:29.369574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:35297 (local address 127.0.0.1:51294)
0811 11:59:29.370518 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:29.370533 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:29.370628 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:29.371410 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:29.371413 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:29.371434 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:29.371537 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:29.371543 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.373225 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.373231 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:29.374486 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:29.374496 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.375361 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.375365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:29.375482 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:29.375521 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 11:59:29.375641 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 11:59:29.375763 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":856,"mutex_wait_us":55,"thread_start_us":98,"threads_started":1}
I20260811 11:59:29.376155 28310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:29.370382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56816 (local address 127.6.143.131:40065)
0811 11:59:29.370842 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:29.370855 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:29.371296 (+ 441us) server_negotiation.cc:407] Connection header received
0811 11:59:29.371419 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:29.371423 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:29.371548 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:29.371693 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:29.371949 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.372652 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.374643 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.375476 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.375584 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:29.375715 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:29.375875 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 11:59:29.376000 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":101,"threads_started":1}
I20260811 11:59:29.376356 28307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:29.370038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51294 (local address 127.6.143.130:35297)
0811 11:59:29.370907 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:29.370918 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:29.370937 (+ 19us) server_negotiation.cc:407] Connection header received
0811 11:59:29.371037 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:29.371043 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:29.371238 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:29.371708 (+ 470us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:29.371873 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.373025 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.374608 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.375222 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.375361 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:29.375520 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:29.376060 (+ 540us) server_negotiation.cc:299] Negotiation successful
0811 11:59:29.376193 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":756,"thread_start_us":169,"threads_started":1}
I20260811 11:59:29.376603 28304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:29.370169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:40065 (local address 127.0.0.1:56816)
0811 11:59:29.371181 (+ 1012us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:29.371193 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:29.371306 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:29.371671 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:29.371674 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:29.371691 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:29.371796 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:29.371806 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.372781 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.372792 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:29.374518 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:29.374524 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.375612 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.375618 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:29.375728 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:29.375751 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 11:59:29.375833 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 11:59:29.375921 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":949,"mutex_wait_us":88,"thread_start_us":95,"threads_started":1}
I20260811 11:59:29.376497 28306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:29.369408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:42577 (local address 127.0.0.1:35466)
0811 11:59:29.370046 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:29.370096 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:29.370212 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:29.371041 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:29.371047 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:29.371074 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:29.371229 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:29.371237 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.372885 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.372888 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:29.374495 (+ 1607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:29.374506 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.375790 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.375794 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:29.375944 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:29.375968 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 11:59:29.376038 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 11:59:29.376355 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"mutex_wait_us":158,"thread_start_us":99,"threads_started":1}
I20260811 11:59:29.376853 28305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:29.369538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35466 (local address 127.6.143.129:42577)
0811 11:59:29.370519 (+ 981us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:29.370537 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:29.370563 (+ 26us) server_negotiation.cc:407] Connection header received
0811 11:59:29.370644 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:29.370650 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:29.370873 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:29.371036 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:29.371394 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.372700 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.374643 (+ 1943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.375476 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.375584 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:29.375715 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:29.376561 (+ 846us) server_negotiation.cc:299] Negotiation successful
0811 11:59:29.376671 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":873,"thread_start_us":87,"threads_started":1}
I20260811 11:59:29.379662 28219 tablet_service.cc:1467] Processing CreateTablet for tablet 5462313aef8d41eabe15ecf1fa176dc8 (DEFAULT_TABLE table=test-table [id=839f00fd16114c55a9c66dcf168ec504]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:59:29.379632 27926 tablet_service.cc:1467] Processing CreateTablet for tablet 5462313aef8d41eabe15ecf1fa176dc8 (DEFAULT_TABLE table=test-table [id=839f00fd16114c55a9c66dcf168ec504]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:59:29.379645 28071 tablet_service.cc:1467] Processing CreateTablet for tablet 5462313aef8d41eabe15ecf1fa176dc8 (DEFAULT_TABLE table=test-table [id=839f00fd16114c55a9c66dcf168ec504]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:59:29.380803 28071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5462313aef8d41eabe15ecf1fa176dc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:29.380798 27926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5462313aef8d41eabe15ecf1fa176dc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:29.380823 28219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5462313aef8d41eabe15ecf1fa176dc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:29.387475 28311 tablet_bootstrap.cc:492] T 5462313aef8d41eabe15ecf1fa176dc8 P 2e6a6dab78334d86b83da60323006260: Bootstrap starting.
I20260811 11:59:29.388180 28312 tablet_bootstrap.cc:492] T 5462313aef8d41eabe15ecf1fa176dc8 P bbfd5fded9194fe0a91aee4b56f04398: Bootstrap starting.
I20260811 11:59:29.389083 28313 tablet_bootstrap.cc:492] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0: Bootstrap starting.
I20260811 11:59:29.390511 28312 tablet_bootstrap.cc:654] T 5462313aef8d41eabe15ecf1fa176dc8 P bbfd5fded9194fe0a91aee4b56f04398: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:29.390573 28311 tablet_bootstrap.cc:654] T 5462313aef8d41eabe15ecf1fa176dc8 P 2e6a6dab78334d86b83da60323006260: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:29.391242 28313 tablet_bootstrap.cc:654] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:29.391579 28311 log.cc:824] T 5462313aef8d41eabe15ecf1fa176dc8 P 2e6a6dab78334d86b83da60323006260: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:29.391831 28312 log.cc:824] T 5462313aef8d41eabe15ecf1fa176dc8 P bbfd5fded9194fe0a91aee4b56f04398: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:29.392230 28313 log.cc:824] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:29.393909 28313 tablet_bootstrap.cc:492] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0: No bootstrap required, opened a new log
I20260811 11:59:29.393937 28311 tablet_bootstrap.cc:492] T 5462313aef8d41eabe15ecf1fa176dc8 P 2e6a6dab78334d86b83da60323006260: No bootstrap required, opened a new log
I20260811 11:59:29.393935 28312 tablet_bootstrap.cc:492] T 5462313aef8d41eabe15ecf1fa176dc8 P bbfd5fded9194fe0a91aee4b56f04398: No bootstrap required, opened a new log
I20260811 11:59:29.394122 28313 ts_tablet_manager.cc:1397] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 11:59:29.394125 28311 ts_tablet_manager.cc:1397] T 5462313aef8d41eabe15ecf1fa176dc8 P 2e6a6dab78334d86b83da60323006260: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 11:59:29.394125 28312 ts_tablet_manager.cc:1397] T 5462313aef8d41eabe15ecf1fa176dc8 P bbfd5fded9194fe0a91aee4b56f04398: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 11:59:29.397146 28313 raft_consensus.cc:348] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd5fded9194fe0a91aee4b56f04398" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42577 } } peers { permanent_uuid: "3e36d8c5a02a4b72b06934a7e4b21fd0" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 35297 } } peers { permanent_uuid: "2e6a6dab78334d86b83da60323006260" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 40065 } }
I20260811 11:59:29.397408 28313 raft_consensus.cc:374] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:29.397483 28313 raft_consensus.cc:729] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e36d8c5a02a4b72b06934a7e4b21fd0, State: Initialized, Role: FOLLOWER
I20260811 11:59:29.398022 28313 consensus_queue.cc:260] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd5fded9194fe0a91aee4b56f04398" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42577 } } peers { permanent_uuid: "3e36d8c5a02a4b72b06934a7e4b21fd0" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 35297 } } peers { permanent_uuid: "2e6a6dab78334d86b83da60323006260" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 40065 } }
I20260811 11:59:29.398535 28312 raft_consensus.cc:348] T 5462313aef8d41eabe15ecf1fa176dc8 P bbfd5fded9194fe0a91aee4b56f04398 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd5fded9194fe0a91aee4b56f04398" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42577 } } peers { permanent_uuid: "3e36d8c5a02a4b72b06934a7e4b21fd0" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 35297 } } peers { permanent_uuid: "2e6a6dab78334d86b83da60323006260" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 40065 } }
I20260811 11:59:29.398535 28311 raft_consensus.cc:348] T 5462313aef8d41eabe15ecf1fa176dc8 P 2e6a6dab78334d86b83da60323006260 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd5fded9194fe0a91aee4b56f04398" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42577 } } peers { permanent_uuid: "3e36d8c5a02a4b72b06934a7e4b21fd0" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 35297 } } peers { permanent_uuid: "2e6a6dab78334d86b83da60323006260" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 40065 } }
I20260811 11:59:29.398813 28312 raft_consensus.cc:374] T 5462313aef8d41eabe15ecf1fa176dc8 P bbfd5fded9194fe0a91aee4b56f04398 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:29.398813 28311 raft_consensus.cc:374] T 5462313aef8d41eabe15ecf1fa176dc8 P 2e6a6dab78334d86b83da60323006260 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:29.398881 28312 raft_consensus.cc:729] T 5462313aef8d41eabe15ecf1fa176dc8 P bbfd5fded9194fe0a91aee4b56f04398 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbfd5fded9194fe0a91aee4b56f04398, State: Initialized, Role: FOLLOWER
I20260811 11:59:29.398881 28311 raft_consensus.cc:729] T 5462313aef8d41eabe15ecf1fa176dc8 P 2e6a6dab78334d86b83da60323006260 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e6a6dab78334d86b83da60323006260, State: Initialized, Role: FOLLOWER
I20260811 11:59:29.399214 28313 ts_tablet_manager.cc:1428] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 11:59:29.399362 28312 consensus_queue.cc:260] T 5462313aef8d41eabe15ecf1fa176dc8 P bbfd5fded9194fe0a91aee4b56f04398 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd5fded9194fe0a91aee4b56f04398" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42577 } } peers { permanent_uuid: "3e36d8c5a02a4b72b06934a7e4b21fd0" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 35297 } } peers { permanent_uuid: "2e6a6dab78334d86b83da60323006260" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 40065 } }
I20260811 11:59:29.399362 28311 consensus_queue.cc:260] T 5462313aef8d41eabe15ecf1fa176dc8 P 2e6a6dab78334d86b83da60323006260 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd5fded9194fe0a91aee4b56f04398" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42577 } } peers { permanent_uuid: "3e36d8c5a02a4b72b06934a7e4b21fd0" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 35297 } } peers { permanent_uuid: "2e6a6dab78334d86b83da60323006260" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 40065 } }
I20260811 11:59:29.399587 28147 heartbeater.cc:503] Master 127.6.143.190:41885 was elected leader, sending a full tablet report...
I20260811 11:59:29.400056 28291 heartbeater.cc:503] Master 127.6.143.190:41885 was elected leader, sending a full tablet report...
I20260811 11:59:29.400501 28311 ts_tablet_manager.cc:1428] T 5462313aef8d41eabe15ecf1fa176dc8 P 2e6a6dab78334d86b83da60323006260: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260811 11:59:29.400501 28312 ts_tablet_manager.cc:1428] T 5462313aef8d41eabe15ecf1fa176dc8 P bbfd5fded9194fe0a91aee4b56f04398: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 11:59:29.401140 28000 heartbeater.cc:503] Master 127.6.143.190:41885 was elected leader, sending a full tablet report...
I20260811 11:59:29.411182 28317 raft_consensus.cc:482] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:59:29.411314 28317 raft_consensus.cc:504] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd5fded9194fe0a91aee4b56f04398" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42577 } } peers { permanent_uuid: "3e36d8c5a02a4b72b06934a7e4b21fd0" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 35297 } } peers { permanent_uuid: "2e6a6dab78334d86b83da60323006260" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 40065 } }
I20260811 11:59:29.412370 28317 leader_election.cc:288] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bbfd5fded9194fe0a91aee4b56f04398 (127.6.143.129:42577), 2e6a6dab78334d86b83da60323006260 (127.6.143.131:40065)
I20260811 11:59:29.415930 28305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:29.412695 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:33801 (local address 127.6.143.129:42577)
0811 11:59:29.412816 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:29.412820 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:29.412852 (+ 32us) server_negotiation.cc:407] Connection header received
0811 11:59:29.412987 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:29.412990 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:29.413044 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:29.413150 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:29.413429 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.413952 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.414977 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.415376 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.415503 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:29.415555 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:29.415735 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 11:59:29.415809 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 11:59:29.415944 28149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:29.412606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:42577 (local address 127.6.143.130:33801)
0811 11:59:29.412761 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:29.412778 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:29.412860 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:29.413164 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:29.413168 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:29.413186 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:29.413313 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:29.413319 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.414069 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.414074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:29.414848 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:29.414857 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.415496 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.415500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:29.415606 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:29.415631 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 11:59:29.415718 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 11:59:29.415787 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 11:59:29.416450 28310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:29.412987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:37561 (local address 127.6.143.131:40065)
0811 11:59:29.413117 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:29.413122 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:29.413391 (+ 269us) server_negotiation.cc:407] Connection header received
0811 11:59:29.413528 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:29.413531 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:29.413577 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:29.413656 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:29.413937 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.414414 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.415437 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.415832 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.416007 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:29.416054 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:29.416288 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 11:59:29.416370 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 11:59:29.416456 28320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:29.412883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:40065 (local address 127.6.143.130:37561)
0811 11:59:29.413307 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:29.413318 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:29.413426 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:29.413694 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:29.413697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:29.413713 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:29.413833 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:29.413839 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.414523 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.414528 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:29.415316 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:29.415325 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.416010 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.416014 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:29.416122 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:29.416145 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 11:59:29.416265 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 11:59:29.416334 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":121,"threads_started":1}
I20260811 11:59:29.416579 27947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5462313aef8d41eabe15ecf1fa176dc8" candidate_uuid: "3e36d8c5a02a4b72b06934a7e4b21fd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbfd5fded9194fe0a91aee4b56f04398" is_pre_election: true
I20260811 11:59:29.416903 27947 raft_consensus.cc:2454] T 5462313aef8d41eabe15ecf1fa176dc8 P bbfd5fded9194fe0a91aee4b56f04398 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e36d8c5a02a4b72b06934a7e4b21fd0 in term 0.
I20260811 11:59:29.417328 28242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5462313aef8d41eabe15ecf1fa176dc8" candidate_uuid: "3e36d8c5a02a4b72b06934a7e4b21fd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e6a6dab78334d86b83da60323006260" is_pre_election: true
I20260811 11:59:29.417439 28023 leader_election.cc:302] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e36d8c5a02a4b72b06934a7e4b21fd0, bbfd5fded9194fe0a91aee4b56f04398; no voters:
I20260811 11:59:29.417629 28242 raft_consensus.cc:2454] T 5462313aef8d41eabe15ecf1fa176dc8 P 2e6a6dab78334d86b83da60323006260 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e36d8c5a02a4b72b06934a7e4b21fd0 in term 0.
I20260811 11:59:29.417742 28317 raft_consensus.cc:2781] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:59:29.417833 28317 raft_consensus.cc:482] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:59:29.417891 28317 raft_consensus.cc:3037] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:29.418927 28317 raft_consensus.cc:504] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd5fded9194fe0a91aee4b56f04398" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42577 } } peers { permanent_uuid: "3e36d8c5a02a4b72b06934a7e4b21fd0" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 35297 } } peers { permanent_uuid: "2e6a6dab78334d86b83da60323006260" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 40065 } }
I20260811 11:59:29.419191 28317 leader_election.cc:288] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0 [CANDIDATE]: Term 1 election: Requested vote from peers bbfd5fded9194fe0a91aee4b56f04398 (127.6.143.129:42577), 2e6a6dab78334d86b83da60323006260 (127.6.143.131:40065)
I20260811 11:59:29.419564 27947 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5462313aef8d41eabe15ecf1fa176dc8" candidate_uuid: "3e36d8c5a02a4b72b06934a7e4b21fd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bbfd5fded9194fe0a91aee4b56f04398"
I20260811 11:59:29.419631 28242 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5462313aef8d41eabe15ecf1fa176dc8" candidate_uuid: "3e36d8c5a02a4b72b06934a7e4b21fd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e6a6dab78334d86b83da60323006260"
I20260811 11:59:29.419693 27947 raft_consensus.cc:3037] T 5462313aef8d41eabe15ecf1fa176dc8 P bbfd5fded9194fe0a91aee4b56f04398 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:29.419756 28242 raft_consensus.cc:3037] T 5462313aef8d41eabe15ecf1fa176dc8 P 2e6a6dab78334d86b83da60323006260 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:29.420698 27947 raft_consensus.cc:2454] T 5462313aef8d41eabe15ecf1fa176dc8 P bbfd5fded9194fe0a91aee4b56f04398 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e36d8c5a02a4b72b06934a7e4b21fd0 in term 1.
I20260811 11:59:29.420837 28242 raft_consensus.cc:2454] T 5462313aef8d41eabe15ecf1fa176dc8 P 2e6a6dab78334d86b83da60323006260 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e36d8c5a02a4b72b06934a7e4b21fd0 in term 1.
I20260811 11:59:29.421053 28023 leader_election.cc:302] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e36d8c5a02a4b72b06934a7e4b21fd0, bbfd5fded9194fe0a91aee4b56f04398; no voters:
I20260811 11:59:29.421233 28317 raft_consensus.cc:2781] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:29.421458 28317 raft_consensus.cc:686] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0 [term 1 LEADER]: Becoming Leader. State: Replica: 3e36d8c5a02a4b72b06934a7e4b21fd0, State: Running, Role: LEADER
I20260811 11:59:29.421764 28317 consensus_queue.cc:237] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd5fded9194fe0a91aee4b56f04398" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42577 } } peers { permanent_uuid: "3e36d8c5a02a4b72b06934a7e4b21fd0" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 35297 } } peers { permanent_uuid: "2e6a6dab78334d86b83da60323006260" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 40065 } }
I20260811 11:59:29.424712 27790 catalog_manager.cc:5495] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e36d8c5a02a4b72b06934a7e4b21fd0 (127.6.143.130). New cstate: current_term: 1 leader_uuid: "3e36d8c5a02a4b72b06934a7e4b21fd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bbfd5fded9194fe0a91aee4b56f04398" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42577 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e36d8c5a02a4b72b06934a7e4b21fd0" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 35297 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2e6a6dab78334d86b83da60323006260" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 40065 } health_report { overall_health: UNKNOWN } } }
I20260811 11:59:29.437829 28307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:29.434641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51304 (local address 127.6.143.130:35297)
0811 11:59:29.434771 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:29.434775 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:29.434787 (+ 12us) server_negotiation.cc:407] Connection header received
0811 11:59:29.434912 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:29.434914 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:29.434963 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:29.435048 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:59:29.435238 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.435713 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.436562 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.436769 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.436849 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:29.437098 (+ 249us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:59:29.437485 (+ 387us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:59:29.437571 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:29.437626 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 11:59:29.437670 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 11:59:29.443338 28242 raft_consensus.cc:1261] T 5462313aef8d41eabe15ecf1fa176dc8 P 2e6a6dab78334d86b83da60323006260 [term 1 FOLLOWER]: Refusing update from remote peer 3e36d8c5a02a4b72b06934a7e4b21fd0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:59:29.443346 27947 raft_consensus.cc:1261] T 5462313aef8d41eabe15ecf1fa176dc8 P bbfd5fded9194fe0a91aee4b56f04398 [term 1 FOLLOWER]: Refusing update from remote peer 3e36d8c5a02a4b72b06934a7e4b21fd0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:59:29.444085 28317 consensus_queue.cc:1035] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bbfd5fded9194fe0a91aee4b56f04398" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 42577 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:59:29.444290 28321 consensus_queue.cc:1035] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e6a6dab78334d86b83da60323006260" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 40065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:59:29.450045 28325 mvcc.cc:205] Tried to move back new op lower bound from 7317297436433453056 to 7317297436354048000. Current Snapshot: MvccSnapshot[applied={T|T < 7317297436433453056}]
I20260811 11:59:29.453873 28327 mvcc.cc:205] Tried to move back new op lower bound from 7317297436433453056 to 7317297436354048000. Current Snapshot: MvccSnapshot[applied={T|T < 7317297436433453056}]
I20260811 11:59:29.459074 27791 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42740:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:59:29.459229 27791 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42740:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:59:29.468262 27791 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c12debeef2bb4384add8f6bd160fb53c: Sending DeleteTablet for 3 replicas of tablet 5462313aef8d41eabe15ecf1fa176dc8
I20260811 11:59:29.469147 27926 tablet_service.cc:1514] Processing DeleteTablet for tablet 5462313aef8d41eabe15ecf1fa176dc8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:59:29 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:35466
I20260811 11:59:29.469201 28219 tablet_service.cc:1514] Processing DeleteTablet for tablet 5462313aef8d41eabe15ecf1fa176dc8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:59:29 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:56816
I20260811 11:59:29.469841 28336 tablet_replica.cc:330] T 5462313aef8d41eabe15ecf1fa176dc8 P 2e6a6dab78334d86b83da60323006260: stopping tablet replica
I20260811 11:59:29.470211 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 27861
I20260811 11:59:29.471647 28071 tablet_service.cc:1514] Processing DeleteTablet for tablet 5462313aef8d41eabe15ecf1fa176dc8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:59:29 UTC) from {username='kudu', principal='kudu/127.6.143.190@KRBTEST.COM'} at 127.0.0.1:51294
I20260811 11:59:29.475732 28336 raft_consensus.cc:2229] T 5462313aef8d41eabe15ecf1fa176dc8 P 2e6a6dab78334d86b83da60323006260 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:59:29.476110 28336 raft_consensus.cc:2258] T 5462313aef8d41eabe15ecf1fa176dc8 P 2e6a6dab78334d86b83da60323006260 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:59:29.477849 28336 ts_tablet_manager.cc:1905] T 5462313aef8d41eabe15ecf1fa176dc8 P 2e6a6dab78334d86b83da60323006260: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:59:29.479936 28337 tablet_replica.cc:330] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0: stopping tablet replica
I20260811 11:59:29.480306 28337 raft_consensus.cc:2229] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0 [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:59:29.480733 28337 raft_consensus.cc:2258] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:59:29.481557 28336 ts_tablet_manager.cc:1918] T 5462313aef8d41eabe15ecf1fa176dc8 P 2e6a6dab78334d86b83da60323006260: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:59:29.481670 28336 log.cc:1196] T 5462313aef8d41eabe15ecf1fa176dc8 P 2e6a6dab78334d86b83da60323006260: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786449319379472-6718-0/minicluster-data/ts-2/wal/wals/5462313aef8d41eabe15ecf1fa176dc8
I20260811 11:59:29.482023 28336 ts_tablet_manager.cc:1939] T 5462313aef8d41eabe15ecf1fa176dc8 P 2e6a6dab78334d86b83da60323006260: Deleting consensus metadata
I20260811 11:59:29.482506 28337 ts_tablet_manager.cc:1905] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 11:59:29.483464 27776 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.143.129:42577 (error 108)
I20260811 11:59:29.484040 27776 catalog_manager.cc:4841] TS 2e6a6dab78334d86b83da60323006260 (127.6.143.131:40065): tablet 5462313aef8d41eabe15ecf1fa176dc8 (table test-table [id=839f00fd16114c55a9c66dcf168ec504]) successfully deleted
W20260811 11:59:29.484687 28306 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:59:29.484359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:42577 (local address 127.0.0.1:35480)
0811 11:59:29.484503 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:29.484580 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.129:42577: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":55}
W20260811 11:59:29.484827 27776 catalog_manager.cc:4584] TS bbfd5fded9194fe0a91aee4b56f04398 (127.6.143.129:42577): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5462313aef8d41eabe15ecf1fa176dc8: Network error: Client connection negotiation failed: client connection to 127.6.143.129:42577: connect: Connection reset by peer (error 104)
I20260811 11:59:29.485195 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 28005
I20260811 11:59:29.485262 28337 ts_tablet_manager.cc:1918] T 5462313aef8d41eabe15ecf1fa176dc8 P 3e36d8c5a02a4b72b06934a7e4b21fd0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:59:29.500155 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 28152
I20260811 11:59:29.500466 28306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:29.500173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:35297 (local address 127.0.0.1:51306)
0811 11:59:29.500315 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:29.500376 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.130:35297: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 11:59:29.507043 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 27757
2026-08-11T11:59:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2301 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T11:59:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:59:29Z Disabled control of system clock
I20260811 11:59:29.539774 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:39679
--webserver_interface=127.6.143.190
--webserver_port=0
--builtin_ntp_servers=127.6.143.148:46113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:39679
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 11:59:29.654318 28342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:29.654670 28342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:29.654747 28342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:29.658851 28342 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:59:29.658937 28342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:29.658967 28342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:29.659008 28342 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:59:29.659042 28342 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:59:29.663697 28342 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:46113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:39679
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:39679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:29.664867 28342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:29.666257 28342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:29.675125 28347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:29.675130 28348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:29.675212 28350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:29.675473 28342 server_base.cc:1034] running on GCE node
I20260811 11:59:29.676111 28342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:29.677152 28342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:29.678464 28342 hybrid_clock.cc:648] HybridClock initialized: now 1786449569678432 us; error 40 us; skew 500 ppm
I20260811 11:59:29.680419 28342 webserver.cc:460] Webserver started at http://127.6.143.190:46681/ using document root <none> and password file <none>
I20260811 11:59:29.681015 28342 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:29.681110 28342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:29.681357 28342 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:29.683429 28342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "7e20095ec04645989208f76b720da428"
format_stamp: "Formatted at 2026-08-11 11:59:29 on dist-test-slave-zkpd"
server_key: "c77d2b767d39bb19ae9d29cbab38690f"
server_key_iv: "a4ba4009302928117022d60d87d8b4b6"
server_key_version: "encryptionkey@0"
I20260811 11:59:29.684078 28342 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "7e20095ec04645989208f76b720da428"
format_stamp: "Formatted at 2026-08-11 11:59:29 on dist-test-slave-zkpd"
server_key: "c77d2b767d39bb19ae9d29cbab38690f"
server_key_iv: "a4ba4009302928117022d60d87d8b4b6"
server_key_version: "encryptionkey@0"
I20260811 11:59:29.687696 28342 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 11:59:29.690110 28357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:29.691304 28342 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:59:29.691448 28342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "7e20095ec04645989208f76b720da428"
format_stamp: "Formatted at 2026-08-11 11:59:29 on dist-test-slave-zkpd"
server_key: "c77d2b767d39bb19ae9d29cbab38690f"
server_key_iv: "a4ba4009302928117022d60d87d8b4b6"
server_key_version: "encryptionkey@0"
I20260811 11:59:29.691555 28342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:29.706522 28342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:29.707121 28342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:29.707309 28342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:29.714458 28342 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:39679
I20260811 11:59:29.714461 28411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:39679 every 8 connection(s)
I20260811 11:59:29.715329 28342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:59:29.718351 28412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:29.723654 28412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e20095ec04645989208f76b720da428: Bootstrap starting.
I20260811 11:59:29.725637 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 28342
I20260811 11:59:29.725785 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:59:29.726078 6718 external_mini_cluster.cc:1428] Setting key ed57015c5713913384b703e181124325
I20260811 11:59:29.726172 28412 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e20095ec04645989208f76b720da428: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:29.727005 28412 log.cc:824] T 00000000000000000000000000000000 P 7e20095ec04645989208f76b720da428: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:29.729240 28412 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e20095ec04645989208f76b720da428: No bootstrap required, opened a new log
I20260811 11:59:29.733489 28412 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e20095ec04645989208f76b720da428 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e20095ec04645989208f76b720da428" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39679 } }
I20260811 11:59:29.733771 28412 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e20095ec04645989208f76b720da428 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:29.733865 28412 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e20095ec04645989208f76b720da428 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e20095ec04645989208f76b720da428, State: Initialized, Role: FOLLOWER
I20260811 11:59:29.734326 28412 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e20095ec04645989208f76b720da428 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e20095ec04645989208f76b720da428" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39679 } }
I20260811 11:59:29.734453 28412 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e20095ec04645989208f76b720da428 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:59:29.734536 28412 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e20095ec04645989208f76b720da428 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:59:29.734658 28412 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e20095ec04645989208f76b720da428 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:29.735625 28412 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e20095ec04645989208f76b720da428 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e20095ec04645989208f76b720da428" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39679 } }
I20260811 11:59:29.735718 28416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:29.727177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34056 (local address 127.6.143.190:39679)
0811 11:59:29.727808 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:29.727829 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:29.727872 (+ 43us) server_negotiation.cc:407] Connection header received
0811 11:59:29.728732 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:29.728755 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:29.729156 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:29.729493 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:29.729916 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.731252 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.732223 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.733032 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.733194 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:29.734052 (+ 858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:29.734057 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:29.734069 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:29.734449 (+ 380us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:29.734542 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:29.734578 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:29.734690 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:29.734806 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 11:59:29.735066 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":106,"threads_started":1}
I20260811 11:59:29.736023 28412 leader_election.cc:302] T 00000000000000000000000000000000 P 7e20095ec04645989208f76b720da428 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e20095ec04645989208f76b720da428; no voters:
I20260811 11:59:29.736361 28412 leader_election.cc:288] T 00000000000000000000000000000000 P 7e20095ec04645989208f76b720da428 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:59:29.736481 28417 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e20095ec04645989208f76b720da428 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:29.736688 28417 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e20095ec04645989208f76b720da428 [term 1 LEADER]: Becoming Leader. State: Replica: 7e20095ec04645989208f76b720da428, State: Running, Role: LEADER
I20260811 11:59:29.736990 28417 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e20095ec04645989208f76b720da428 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e20095ec04645989208f76b720da428" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39679 } }
I20260811 11:59:29.737812 28412 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e20095ec04645989208f76b720da428 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:59:29.738226 28418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e20095ec04645989208f76b720da428 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e20095ec04645989208f76b720da428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e20095ec04645989208f76b720da428" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39679 } } }
I20260811 11:59:29.738287 28419 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e20095ec04645989208f76b720da428 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e20095ec04645989208f76b720da428. Latest consensus state: current_term: 1 leader_uuid: "7e20095ec04645989208f76b720da428" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e20095ec04645989208f76b720da428" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 39679 } } }
I20260811 11:59:29.738348 28418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e20095ec04645989208f76b720da428 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:29.738584 28419 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e20095ec04645989208f76b720da428 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:29.738819 28421 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:59:29.741581 28421 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:59:29.746613 28421 catalog_manager.cc:1347] Generated new cluster ID: 3d21ad4ed40d4eb9af8407c3934f9a5d
I20260811 11:59:29.746676 28421 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:59:29.769014 28421 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:59:29.770215 28421 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:59:29.776085 28421 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e20095ec04645989208f76b720da428: Generated new TSK 0
I20260811 11:59:29.776777 28421 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:59:29.780602 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.129:0
--local_ip_for_outbound_sockets=127.6.143.129
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:39679
--builtin_ntp_servers=127.6.143.148:46113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 11:59:29.895090 28436 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:29.895378 28436 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:29.895421 28436 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:29.900018 28436 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:29.900065 28436 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:29.900117 28436 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.129
I20260811 11:59:29.904670 28436 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:46113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.143.129
--webserver_port=0
--tserver_master_addrs=127.6.143.190:39679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28436
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.129
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:29.905539 28436 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:29.906796 28436 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:29.916328 28442 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:29.916386 28444 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:29.916327 28441 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:29.916570 28436 server_base.cc:1034] running on GCE node
I20260811 11:59:29.917016 28436 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:29.917590 28436 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:29.918814 28436 hybrid_clock.cc:648] HybridClock initialized: now 1786449569918795 us; error 29 us; skew 500 ppm
I20260811 11:59:29.920722 28436 webserver.cc:460] Webserver started at http://127.6.143.129:38247/ using document root <none> and password file <none>
I20260811 11:59:29.921276 28436 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:29.921339 28436 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:29.921588 28436 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:29.923552 28436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-0/data/instance:
uuid: "208e1a5456a14e18883a8acffe844ab9"
format_stamp: "Formatted at 2026-08-11 11:59:29 on dist-test-slave-zkpd"
server_key: "4c81b18b3510da735c40e6ab131ead96"
server_key_iv: "526cd14992dae94562d86f043aae1931"
server_key_version: "encryptionkey@0"
I20260811 11:59:29.924134 28436 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance:
uuid: "208e1a5456a14e18883a8acffe844ab9"
format_stamp: "Formatted at 2026-08-11 11:59:29 on dist-test-slave-zkpd"
server_key: "4c81b18b3510da735c40e6ab131ead96"
server_key_iv: "526cd14992dae94562d86f043aae1931"
server_key_version: "encryptionkey@0"
I20260811 11:59:29.927671 28436 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 11:59:29.930090 28450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:29.931288 28436 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:59:29.931420 28436 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-0/wal
uuid: "208e1a5456a14e18883a8acffe844ab9"
format_stamp: "Formatted at 2026-08-11 11:59:29 on dist-test-slave-zkpd"
server_key: "4c81b18b3510da735c40e6ab131ead96"
server_key_iv: "526cd14992dae94562d86f043aae1931"
server_key_version: "encryptionkey@0"
I20260811 11:59:29.931527 28436 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:29.941390 28436 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:29.941982 28436 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:29.942159 28436 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:29.942701 28436 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:59:29.943567 28436 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:59:29.943650 28436 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:29.943737 28436 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:59:29.943785 28436 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:29.952504 28436 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.129:45661
I20260811 11:59:29.952534 28570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.129:45661 every 8 connection(s)
I20260811 11:59:29.953295 28436 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-0/data/info.pb
I20260811 11:59:29.957620 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 28436
I20260811 11:59:29.957789 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-0/wal/instance
I20260811 11:59:29.958093 6718 external_mini_cluster.cc:1428] Setting key 66ab9ba11f3af059766acc81393487bc
I20260811 11:59:29.960182 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.130:0
--local_ip_for_outbound_sockets=127.6.143.130
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:39679
--builtin_ntp_servers=127.6.143.148:46113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 11:59:29.962500 28416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:29.955036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.129:56709 (local address 127.6.143.190:39679)
0811 11:59:29.955160 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:29.955163 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:29.956137 (+ 974us) server_negotiation.cc:407] Connection header received
0811 11:59:29.957515 (+ 1378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:29.957519 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:29.957585 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:29.957690 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:29.958629 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.959228 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.960554 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.960805 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.960919 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:29.962006 (+ 1087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:29.962015 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:29.962018 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:29.962029 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:29.962046 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:29.962054 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:29.962135 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:29.962294 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 11:59:29.962356 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 11:59:29.963266 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:29.955332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39679 (local address 127.6.143.129:56709)
0811 11:59:29.956003 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:29.956042 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:29.957295 (+ 1253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:29.957887 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:29.957898 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:29.957998 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:59:29.958491 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:29.958504 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.959368 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.959371 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:29.960436 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:29.960447 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:29.960977 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:29.960982 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:29.961137 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:29.961749 (+ 612us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:59:29.961751 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:59:29.961787 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:59:29.961845 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:59:29.962137 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:59:29.962150 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:59:29.962158 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 11:59:29.962290 (+ 132us) client_negotiation.cc:240] Negotiation successful
0811 11:59:29.962554 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":107,"threads_started":1}
I20260811 11:59:29.964524 28571 heartbeater.cc:347] Connected to a master server at 127.6.143.190:39679
I20260811 11:59:29.964748 28571 heartbeater.cc:464] Registering TS with master...
I20260811 11:59:29.965240 28571 heartbeater.cc:511] Master 127.6.143.190:39679 requested a full tablet report, sending...
I20260811 11:59:29.966943 28375 ts_manager.cc:194] Registered new tserver with Master: 208e1a5456a14e18883a8acffe844ab9 (127.6.143.129:45661)
I20260811 11:59:29.968801 28375 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.143.129:56709
W20260811 11:59:30.074492 28574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:30.074725 28574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:30.074761 28574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:30.078670 28574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:30.078722 28574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:30.078771 28574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.130
I20260811 11:59:30.083294 28574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:46113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.143.130
--webserver_port=0
--tserver_master_addrs=127.6.143.190:39679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.130
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:30.084220 28574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:30.085501 28574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:30.095322 28579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:30.095376 28574 server_base.cc:1034] running on GCE node
W20260811 11:59:30.095299 28580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:30.095304 28582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:30.096073 28574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:30.096653 28574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:30.097810 28574 hybrid_clock.cc:648] HybridClock initialized: now 1786449570097783 us; error 43 us; skew 500 ppm
I20260811 11:59:30.103438 28574 webserver.cc:460] Webserver started at http://127.6.143.130:46477/ using document root <none> and password file <none>
I20260811 11:59:30.104099 28574 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:30.104166 28574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:30.104378 28574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:30.106426 28574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-1/data/instance:
uuid: "c06946c9100d40c7a2c4d0390902ac77"
format_stamp: "Formatted at 2026-08-11 11:59:30 on dist-test-slave-zkpd"
server_key: "ae5372981e17f507dd7dcd2d8390be42"
server_key_iv: "0fe004b1f0af0b5adfd615b8e769059e"
server_key_version: "encryptionkey@0"
I20260811 11:59:30.106984 28574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance:
uuid: "c06946c9100d40c7a2c4d0390902ac77"
format_stamp: "Formatted at 2026-08-11 11:59:30 on dist-test-slave-zkpd"
server_key: "ae5372981e17f507dd7dcd2d8390be42"
server_key_iv: "0fe004b1f0af0b5adfd615b8e769059e"
server_key_version: "encryptionkey@0"
I20260811 11:59:30.110685 28574 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.002s
I20260811 11:59:30.113330 28589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:30.114530 28574 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 11:59:30.114665 28574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-1/wal
uuid: "c06946c9100d40c7a2c4d0390902ac77"
format_stamp: "Formatted at 2026-08-11 11:59:30 on dist-test-slave-zkpd"
server_key: "ae5372981e17f507dd7dcd2d8390be42"
server_key_iv: "0fe004b1f0af0b5adfd615b8e769059e"
server_key_version: "encryptionkey@0"
I20260811 11:59:30.114773 28574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:30.139810 28574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:30.140503 28574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:30.140689 28574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:30.141244 28574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:59:30.142161 28574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:59:30.142230 28574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:30.142303 28574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:59:30.142345 28574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:30.151244 28574 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.130:34525
I20260811 11:59:30.151276 28708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.130:34525 every 8 connection(s)
I20260811 11:59:30.152129 28574 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-1/data/info.pb
I20260811 11:59:30.158108 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 28574
I20260811 11:59:30.158293 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-1/wal/instance
I20260811 11:59:30.158601 6718 external_mini_cluster.cc:1428] Setting key 847958b2343ddf2df757e707a9ba9468
I20260811 11:59:30.160804 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.143.131:0
--local_ip_for_outbound_sockets=127.6.143.131
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:39679
--builtin_ntp_servers=127.6.143.148:46113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 11:59:30.161664 28416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:30.153976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.130:37419 (local address 127.6.143.190:39679)
0811 11:59:30.154129 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:30.154134 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:30.155221 (+ 1087us) server_negotiation.cc:407] Connection header received
0811 11:59:30.156651 (+ 1430us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:30.156654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:30.156701 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:30.156774 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:30.157619 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.158145 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.159514 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.159737 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.159846 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:30.161184 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:30.161187 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:30.161189 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:30.161203 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:30.161225 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:30.161234 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:30.161304 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:30.161502 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 11:59:30.161555 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 11:59:30.162840 28711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:30.154394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39679 (local address 127.6.143.130:37419)
0811 11:59:30.155074 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:30.155113 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:30.156462 (+ 1349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:30.156935 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:30.156941 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:30.157019 (+ 78us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:59:30.157494 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:30.157506 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.158276 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.158280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:30.159375 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:30.159386 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.159929 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.159936 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:30.160146 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:30.160798 (+ 652us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:59:30.160800 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:59:30.160856 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:59:30.160946 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:59:30.161328 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:59:30.161341 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:59:30.161351 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 11:59:30.161495 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 11:59:30.161799 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":118,"threads_started":1}
I20260811 11:59:30.163995 28709 heartbeater.cc:347] Connected to a master server at 127.6.143.190:39679
I20260811 11:59:30.164245 28709 heartbeater.cc:464] Registering TS with master...
I20260811 11:59:30.164752 28709 heartbeater.cc:511] Master 127.6.143.190:39679 requested a full tablet report, sending...
I20260811 11:59:30.165807 28375 ts_manager.cc:194] Registered new tserver with Master: c06946c9100d40c7a2c4d0390902ac77 (127.6.143.130:34525)
I20260811 11:59:30.166369 28375 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.143.130:37419
W20260811 11:59:30.274711 28712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:30.275010 28712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:30.275092 28712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:30.279173 28712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:30.279247 28712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:30.279337 28712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.143.131
I20260811 11:59:30.284054 28712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:46113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.143.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.143.131
--webserver_port=0
--tserver_master_addrs=127.6.143.190:39679
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.143.131
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:30.285066 28712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:30.286427 28712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 11:59:30.296022 28712 server_base.cc:1034] running on GCE node
W20260811 11:59:30.295919 28720 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:30.295921 28718 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:30.295915 28717 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:30.296695 28712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:30.297319 28712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:30.298482 28712 hybrid_clock.cc:648] HybridClock initialized: now 1786449570298463 us; error 28 us; skew 500 ppm
I20260811 11:59:30.300386 28712 webserver.cc:460] Webserver started at http://127.6.143.131:33425/ using document root <none> and password file <none>
I20260811 11:59:30.300968 28712 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:30.301031 28712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:30.301278 28712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:30.303294 28712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-2/data/instance:
uuid: "80d90edfd12c42b99570288ea7279f14"
format_stamp: "Formatted at 2026-08-11 11:59:30 on dist-test-slave-zkpd"
server_key: "c93c4d7354d97c103d00c3313728a892"
server_key_iv: "fbaca497b6ddc118bcc93c78feca9b0a"
server_key_version: "encryptionkey@0"
I20260811 11:59:30.303936 28712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance:
uuid: "80d90edfd12c42b99570288ea7279f14"
format_stamp: "Formatted at 2026-08-11 11:59:30 on dist-test-slave-zkpd"
server_key: "c93c4d7354d97c103d00c3313728a892"
server_key_iv: "fbaca497b6ddc118bcc93c78feca9b0a"
server_key_version: "encryptionkey@0"
I20260811 11:59:30.307534 28712 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 11:59:30.309969 28728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:30.311134 28712 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:59:30.311272 28712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-2/wal
uuid: "80d90edfd12c42b99570288ea7279f14"
format_stamp: "Formatted at 2026-08-11 11:59:30 on dist-test-slave-zkpd"
server_key: "c93c4d7354d97c103d00c3313728a892"
server_key_iv: "fbaca497b6ddc118bcc93c78feca9b0a"
server_key_version: "encryptionkey@0"
I20260811 11:59:30.311383 28712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:30.326027 28712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:30.326656 28712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:30.326851 28712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:30.327407 28712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 11:59:30.328459 28712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 11:59:30.328532 28712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:30.328604 28712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 11:59:30.328646 28712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:30.337721 28712 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.131:42265
I20260811 11:59:30.337751 28849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.131:42265 every 8 connection(s)
I20260811 11:59:30.338755 28712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-2/data/info.pb
I20260811 11:59:30.348462 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 28712
I20260811 11:59:30.348481 28416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:30.340760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:35675 (local address 127.6.143.190:39679)
0811 11:59:30.340914 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:30.340917 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:30.341999 (+ 1082us) server_negotiation.cc:407] Connection header received
0811 11:59:30.343487 (+ 1488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:30.343490 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:30.343553 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:30.343671 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:30.344637 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.345175 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.346375 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.346613 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.346732 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:30.347878 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:30.347881 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:30.347883 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:30.347946 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:30.347969 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:30.347985 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:30.348066 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:30.348248 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 11:59:30.348321 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 11:59:30.348634 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-2/wal/instance
I20260811 11:59:30.348971 6718 external_mini_cluster.cc:1428] Setting key e31667597ef3563a172ae91b1d0282b8
I20260811 11:59:30.349423 28852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:30.341179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:39679 (local address 127.6.143.131:35675)
0811 11:59:30.341837 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:30.341877 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:30.343243 (+ 1366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:30.343870 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:30.343877 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:30.343989 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:59:30.344482 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:30.344494 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.345315 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.345323 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:30.346238 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:30.346248 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.346780 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.346784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:30.346958 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:30.347602 (+ 644us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 11:59:30.347603 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:59:30.347644 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 11:59:30.347708 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:59:30.348081 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:59:30.348092 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:59:30.348105 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 11:59:30.348240 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 11:59:30.348537 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":123,"threads_started":1}
I20260811 11:59:30.350699 28850 heartbeater.cc:347] Connected to a master server at 127.6.143.190:39679
I20260811 11:59:30.350961 28850 heartbeater.cc:464] Registering TS with master...
I20260811 11:59:30.351509 28850 heartbeater.cc:511] Master 127.6.143.190:39679 requested a full tablet report, sending...
I20260811 11:59:30.352583 28375 ts_manager.cc:194] Registered new tserver with Master: 80d90edfd12c42b99570288ea7279f14 (127.6.143.131:42265)
I20260811 11:59:30.353111 28375 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.143.131:35675
I20260811 11:59:30.363204 6718 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 11:59:30.368918 28416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:30.365293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52940 (local address 127.6.143.190:39679)
0811 11:59:30.365437 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:30.365441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:30.365653 (+ 212us) server_negotiation.cc:407] Connection header received
0811 11:59:30.365750 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:30.365752 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:30.365792 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:30.365858 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:30.366094 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.366596 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.367420 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.367624 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.367701 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:30.368569 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:30.368572 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:30.368574 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:30.368585 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:30.368600 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:30.368607 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:30.368665 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:30.368758 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 11:59:30.368808 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 11:59:30.372006 28375 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52940:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 11:59:30.375119 28375 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 11:59:30.393870 28861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:30.384898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:34525 (local address 127.0.0.1:55428)
0811 11:59:30.386266 (+ 1368us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:30.386321 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:30.386457 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:30.387525 (+ 1068us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:30.387534 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:30.387568 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:30.387729 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:30.387736 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.389286 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.389290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:30.390858 (+ 1568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:30.390869 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.393437 (+ 2568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.393440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:30.393526 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:30.393561 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 11:59:30.393639 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 11:59:30.393751 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1161,"thread_start_us":87,"threads_started":1}
I20260811 11:59:30.394004 28866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:30.385909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55428 (local address 127.6.143.130:34525)
0811 11:59:30.387050 (+ 1141us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:30.387061 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:30.387079 (+ 18us) server_negotiation.cc:407] Connection header received
0811 11:59:30.387142 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:30.387146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:30.387335 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:30.387503 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:30.387879 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.389104 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.392441 (+ 3337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.393301 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.393406 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:30.393613 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:30.393716 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 11:59:30.393824 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1042,"thread_start_us":179,"threads_started":1}
I20260811 11:59:30.394364 28862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:30.385969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:45661 (local address 127.0.0.1:41102)
0811 11:59:30.387743 (+ 1774us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:30.387754 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:30.387879 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:30.388704 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:30.388707 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:30.388724 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:30.388802 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:30.388807 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.391687 (+ 2880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.391690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:30.392880 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:30.392892 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.393957 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.393964 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:30.394056 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:30.394084 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 11:59:30.394152 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 11:59:30.394229 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1707,"thread_start_us":111,"threads_started":1}
I20260811 11:59:30.395004 28867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:30.386431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41102 (local address 127.6.143.129:45661)
0811 11:59:30.386834 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:30.386849 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:30.388114 (+ 1265us) server_negotiation.cc:407] Connection header received
0811 11:59:30.388212 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:30.388218 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:30.388418 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:30.388563 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:30.390378 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.391537 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.393025 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.393823 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.393935 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:30.394676 (+ 741us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:30.394732 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 11:59:30.394826 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":125,"threads_started":1}
I20260811 11:59:30.395656 28865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:30.385078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.131:42265 (local address 127.0.0.1:43488)
0811 11:59:30.386719 (+ 1641us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:30.386732 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:30.386820 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:30.387877 (+ 1057us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:30.387881 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:30.387930 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:30.388044 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:30.388054 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.390333 (+ 2279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.390338 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:30.391544 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:30.391561 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.395211 (+ 3650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.395214 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:30.395310 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:30.395335 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 11:59:30.395466 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 11:59:30.395536 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1557,"thread_start_us":88,"threads_started":1}
I20260811 11:59:30.396044 28863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:30.385284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43488 (local address 127.6.143.131:42265)
0811 11:59:30.386981 (+ 1697us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:30.386994 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:30.387020 (+ 26us) server_negotiation.cc:407] Connection header received
0811 11:59:30.387104 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:30.387110 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:30.387335 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:30.387503 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:30.388929 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.390211 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.391698 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.394848 (+ 3150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.394954 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:30.395138 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:30.395569 (+ 431us) server_negotiation.cc:299] Negotiation successful
0811 11:59:30.395674 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1580,"thread_start_us":104,"threads_started":1}
I20260811 11:59:30.397365 28638 tablet_service.cc:1467] Processing CreateTablet for tablet 5fb7352559844e7aa4996ebbfaa12de8 (DEFAULT_TABLE table=test-table [id=9267da0fe39b4987ae2cb78094fd5442]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:59:30.397568 28501 tablet_service.cc:1467] Processing CreateTablet for tablet 5fb7352559844e7aa4996ebbfaa12de8 (DEFAULT_TABLE table=test-table [id=9267da0fe39b4987ae2cb78094fd5442]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:59:30.398416 28501 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fb7352559844e7aa4996ebbfaa12de8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:30.398416 28638 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fb7352559844e7aa4996ebbfaa12de8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:30.402745 28777 tablet_service.cc:1467] Processing CreateTablet for tablet 5fb7352559844e7aa4996ebbfaa12de8 (DEFAULT_TABLE table=test-table [id=9267da0fe39b4987ae2cb78094fd5442]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 11:59:30.403811 28777 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5fb7352559844e7aa4996ebbfaa12de8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:30.404994 28868 tablet_bootstrap.cc:492] T 5fb7352559844e7aa4996ebbfaa12de8 P c06946c9100d40c7a2c4d0390902ac77: Bootstrap starting.
I20260811 11:59:30.406275 28869 tablet_bootstrap.cc:492] T 5fb7352559844e7aa4996ebbfaa12de8 P 208e1a5456a14e18883a8acffe844ab9: Bootstrap starting.
I20260811 11:59:30.407153 28868 tablet_bootstrap.cc:654] T 5fb7352559844e7aa4996ebbfaa12de8 P c06946c9100d40c7a2c4d0390902ac77: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:30.407874 28868 log.cc:824] T 5fb7352559844e7aa4996ebbfaa12de8 P c06946c9100d40c7a2c4d0390902ac77: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:30.408459 28869 tablet_bootstrap.cc:654] T 5fb7352559844e7aa4996ebbfaa12de8 P 208e1a5456a14e18883a8acffe844ab9: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:30.409289 28869 log.cc:824] T 5fb7352559844e7aa4996ebbfaa12de8 P 208e1a5456a14e18883a8acffe844ab9: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:30.410497 28868 tablet_bootstrap.cc:492] T 5fb7352559844e7aa4996ebbfaa12de8 P c06946c9100d40c7a2c4d0390902ac77: No bootstrap required, opened a new log
I20260811 11:59:30.410704 28868 ts_tablet_manager.cc:1397] T 5fb7352559844e7aa4996ebbfaa12de8 P c06946c9100d40c7a2c4d0390902ac77: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260811 11:59:30.410984 28872 tablet_bootstrap.cc:492] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14: Bootstrap starting.
I20260811 11:59:30.412048 28869 tablet_bootstrap.cc:492] T 5fb7352559844e7aa4996ebbfaa12de8 P 208e1a5456a14e18883a8acffe844ab9: No bootstrap required, opened a new log
I20260811 11:59:30.412242 28869 ts_tablet_manager.cc:1397] T 5fb7352559844e7aa4996ebbfaa12de8 P 208e1a5456a14e18883a8acffe844ab9: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.002s
I20260811 11:59:30.413020 28872 tablet_bootstrap.cc:654] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:30.413708 28872 log.cc:824] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:30.414848 28868 raft_consensus.cc:348] T 5fb7352559844e7aa4996ebbfaa12de8 P c06946c9100d40c7a2c4d0390902ac77 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c06946c9100d40c7a2c4d0390902ac77" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 34525 } } peers { permanent_uuid: "80d90edfd12c42b99570288ea7279f14" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42265 } } peers { permanent_uuid: "208e1a5456a14e18883a8acffe844ab9" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45661 } }
I20260811 11:59:30.415134 28868 raft_consensus.cc:374] T 5fb7352559844e7aa4996ebbfaa12de8 P c06946c9100d40c7a2c4d0390902ac77 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:30.415194 28868 raft_consensus.cc:729] T 5fb7352559844e7aa4996ebbfaa12de8 P c06946c9100d40c7a2c4d0390902ac77 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c06946c9100d40c7a2c4d0390902ac77, State: Initialized, Role: FOLLOWER
I20260811 11:59:30.415704 28872 tablet_bootstrap.cc:492] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14: No bootstrap required, opened a new log
I20260811 11:59:30.415668 28868 consensus_queue.cc:260] T 5fb7352559844e7aa4996ebbfaa12de8 P c06946c9100d40c7a2c4d0390902ac77 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c06946c9100d40c7a2c4d0390902ac77" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 34525 } } peers { permanent_uuid: "80d90edfd12c42b99570288ea7279f14" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42265 } } peers { permanent_uuid: "208e1a5456a14e18883a8acffe844ab9" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45661 } }
I20260811 11:59:30.415877 28872 ts_tablet_manager.cc:1397] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260811 11:59:30.416435 28869 raft_consensus.cc:348] T 5fb7352559844e7aa4996ebbfaa12de8 P 208e1a5456a14e18883a8acffe844ab9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c06946c9100d40c7a2c4d0390902ac77" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 34525 } } peers { permanent_uuid: "80d90edfd12c42b99570288ea7279f14" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42265 } } peers { permanent_uuid: "208e1a5456a14e18883a8acffe844ab9" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45661 } }
I20260811 11:59:30.416689 28869 raft_consensus.cc:374] T 5fb7352559844e7aa4996ebbfaa12de8 P 208e1a5456a14e18883a8acffe844ab9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:30.416726 28868 ts_tablet_manager.cc:1428] T 5fb7352559844e7aa4996ebbfaa12de8 P c06946c9100d40c7a2c4d0390902ac77: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260811 11:59:30.416751 28869 raft_consensus.cc:729] T 5fb7352559844e7aa4996ebbfaa12de8 P 208e1a5456a14e18883a8acffe844ab9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 208e1a5456a14e18883a8acffe844ab9, State: Initialized, Role: FOLLOWER
I20260811 11:59:30.417114 28709 heartbeater.cc:503] Master 127.6.143.190:39679 was elected leader, sending a full tablet report...
I20260811 11:59:30.417213 28869 consensus_queue.cc:260] T 5fb7352559844e7aa4996ebbfaa12de8 P 208e1a5456a14e18883a8acffe844ab9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c06946c9100d40c7a2c4d0390902ac77" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 34525 } } peers { permanent_uuid: "80d90edfd12c42b99570288ea7279f14" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42265 } } peers { permanent_uuid: "208e1a5456a14e18883a8acffe844ab9" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45661 } }
I20260811 11:59:30.417858 28571 heartbeater.cc:503] Master 127.6.143.190:39679 was elected leader, sending a full tablet report...
I20260811 11:59:30.418202 28869 ts_tablet_manager.cc:1428] T 5fb7352559844e7aa4996ebbfaa12de8 P 208e1a5456a14e18883a8acffe844ab9: Time spent starting tablet: real 0.006s user 0.004s sys 0.002s
I20260811 11:59:30.420969 28872 raft_consensus.cc:348] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c06946c9100d40c7a2c4d0390902ac77" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 34525 } } peers { permanent_uuid: "80d90edfd12c42b99570288ea7279f14" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42265 } } peers { permanent_uuid: "208e1a5456a14e18883a8acffe844ab9" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45661 } }
I20260811 11:59:30.421252 28872 raft_consensus.cc:374] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:30.421319 28872 raft_consensus.cc:729] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80d90edfd12c42b99570288ea7279f14, State: Initialized, Role: FOLLOWER
I20260811 11:59:30.421728 28872 consensus_queue.cc:260] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c06946c9100d40c7a2c4d0390902ac77" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 34525 } } peers { permanent_uuid: "80d90edfd12c42b99570288ea7279f14" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42265 } } peers { permanent_uuid: "208e1a5456a14e18883a8acffe844ab9" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45661 } }
I20260811 11:59:30.422410 28850 heartbeater.cc:503] Master 127.6.143.190:39679 was elected leader, sending a full tablet report...
I20260811 11:59:30.422676 28872 ts_tablet_manager.cc:1428] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
W20260811 11:59:30.454505 28572 tablet.cc:2369] T 5fb7352559844e7aa4996ebbfaa12de8 P 208e1a5456a14e18883a8acffe844ab9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:59:30.575325 28876 raft_consensus.cc:482] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 11:59:30.575562 28876 raft_consensus.cc:504] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c06946c9100d40c7a2c4d0390902ac77" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 34525 } } peers { permanent_uuid: "80d90edfd12c42b99570288ea7279f14" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42265 } } peers { permanent_uuid: "208e1a5456a14e18883a8acffe844ab9" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45661 } }
I20260811 11:59:30.576831 28876 leader_election.cc:288] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c06946c9100d40c7a2c4d0390902ac77 (127.6.143.130:34525), 208e1a5456a14e18883a8acffe844ab9 (127.6.143.129:45661)
I20260811 11:59:30.582029 28852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:30.577089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:45661 (local address 127.6.143.131:36729)
0811 11:59:30.577289 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:30.577304 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:30.577418 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:30.577736 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:30.577741 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:30.577760 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:30.577890 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:30.577897 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.578979 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.578984 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:30.580409 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:30.580416 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.581347 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.581351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:30.581590 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:30.581648 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 11:59:30.581747 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 11:59:30.581848 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":194}
I20260811 11:59:30.582067 28866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:30.577189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:45023 (local address 127.6.143.130:34525)
0811 11:59:30.577324 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:30.577331 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:30.577642 (+ 311us) server_negotiation.cc:407] Connection header received
0811 11:59:30.577731 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:30.577735 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:30.577785 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:30.577863 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:30.578174 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.578914 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.580571 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.581172 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.581315 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:30.581369 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:30.581849 (+ 480us) server_negotiation.cc:299] Negotiation successful
0811 11:59:30.581940 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 11:59:30.582067 28867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:30.577245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.143.131:36729 (local address 127.6.143.129:45661)
0811 11:59:30.577384 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:30.577388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:30.577406 (+ 18us) server_negotiation.cc:407] Connection header received
0811 11:59:30.577541 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:30.577543 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:30.577589 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:30.577698 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 11:59:30.578017 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.578847 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.580571 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.581178 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.581320 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:30.581369 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:30.581865 (+ 496us) server_negotiation.cc:299] Negotiation successful
0811 11:59:30.581941 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 11:59:30.582041 28877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:30.577094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.130:34525 (local address 127.6.143.131:45023)
0811 11:59:30.577536 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:30.577555 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:30.577633 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:30.577905 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:30.577908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:30.577925 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 11:59:30.578058 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:30.578064 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.579017 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.579020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:30.580404 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:30.580416 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.581347 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.581351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:30.581621 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:30.581658 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 11:59:30.581773 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 11:59:30.581848 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"mutex_wait_us":180,"thread_start_us":144,"threads_started":1}
I20260811 11:59:30.583014 28522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fb7352559844e7aa4996ebbfaa12de8" candidate_uuid: "80d90edfd12c42b99570288ea7279f14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "208e1a5456a14e18883a8acffe844ab9" is_pre_election: true
I20260811 11:59:30.583020 28661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fb7352559844e7aa4996ebbfaa12de8" candidate_uuid: "80d90edfd12c42b99570288ea7279f14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c06946c9100d40c7a2c4d0390902ac77" is_pre_election: true
I20260811 11:59:30.583355 28522 raft_consensus.cc:2454] T 5fb7352559844e7aa4996ebbfaa12de8 P 208e1a5456a14e18883a8acffe844ab9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80d90edfd12c42b99570288ea7279f14 in term 0.
I20260811 11:59:30.583355 28661 raft_consensus.cc:2454] T 5fb7352559844e7aa4996ebbfaa12de8 P c06946c9100d40c7a2c4d0390902ac77 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80d90edfd12c42b99570288ea7279f14 in term 0.
I20260811 11:59:30.583946 28731 leader_election.cc:302] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 208e1a5456a14e18883a8acffe844ab9, 80d90edfd12c42b99570288ea7279f14, c06946c9100d40c7a2c4d0390902ac77; no voters:
I20260811 11:59:30.584283 28876 raft_consensus.cc:2781] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 11:59:30.584370 28876 raft_consensus.cc:482] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 11:59:30.584403 28876 raft_consensus.cc:3037] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:30.585476 28876 raft_consensus.cc:504] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c06946c9100d40c7a2c4d0390902ac77" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 34525 } } peers { permanent_uuid: "80d90edfd12c42b99570288ea7279f14" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42265 } } peers { permanent_uuid: "208e1a5456a14e18883a8acffe844ab9" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45661 } }
I20260811 11:59:30.585843 28876 leader_election.cc:288] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14 [CANDIDATE]: Term 1 election: Requested vote from peers c06946c9100d40c7a2c4d0390902ac77 (127.6.143.130:34525), 208e1a5456a14e18883a8acffe844ab9 (127.6.143.129:45661)
I20260811 11:59:30.586241 28661 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fb7352559844e7aa4996ebbfaa12de8" candidate_uuid: "80d90edfd12c42b99570288ea7279f14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c06946c9100d40c7a2c4d0390902ac77"
I20260811 11:59:30.586377 28522 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5fb7352559844e7aa4996ebbfaa12de8" candidate_uuid: "80d90edfd12c42b99570288ea7279f14" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "208e1a5456a14e18883a8acffe844ab9"
I20260811 11:59:30.586416 28661 raft_consensus.cc:3037] T 5fb7352559844e7aa4996ebbfaa12de8 P c06946c9100d40c7a2c4d0390902ac77 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:30.586509 28522 raft_consensus.cc:3037] T 5fb7352559844e7aa4996ebbfaa12de8 P 208e1a5456a14e18883a8acffe844ab9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:30.587749 28522 raft_consensus.cc:2454] T 5fb7352559844e7aa4996ebbfaa12de8 P 208e1a5456a14e18883a8acffe844ab9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80d90edfd12c42b99570288ea7279f14 in term 1.
I20260811 11:59:30.587749 28661 raft_consensus.cc:2454] T 5fb7352559844e7aa4996ebbfaa12de8 P c06946c9100d40c7a2c4d0390902ac77 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80d90edfd12c42b99570288ea7279f14 in term 1.
I20260811 11:59:30.588148 28731 leader_election.cc:302] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80d90edfd12c42b99570288ea7279f14, c06946c9100d40c7a2c4d0390902ac77; no voters:
I20260811 11:59:30.588352 28876 raft_consensus.cc:2781] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:30.588603 28876 raft_consensus.cc:686] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14 [term 1 LEADER]: Becoming Leader. State: Replica: 80d90edfd12c42b99570288ea7279f14, State: Running, Role: LEADER
I20260811 11:59:30.588984 28876 consensus_queue.cc:237] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c06946c9100d40c7a2c4d0390902ac77" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 34525 } } peers { permanent_uuid: "80d90edfd12c42b99570288ea7279f14" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42265 } } peers { permanent_uuid: "208e1a5456a14e18883a8acffe844ab9" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45661 } }
W20260811 11:59:30.589751 28851 tablet.cc:2369] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 11:59:30.592466 28375 catalog_manager.cc:5495] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14 reported cstate change: term changed from 0 to 1, leader changed from <none> to 80d90edfd12c42b99570288ea7279f14 (127.6.143.131). New cstate: current_term: 1 leader_uuid: "80d90edfd12c42b99570288ea7279f14" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c06946c9100d40c7a2c4d0390902ac77" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 34525 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80d90edfd12c42b99570288ea7279f14" member_type: VOTER last_known_addr { host: "127.6.143.131" port: 42265 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "208e1a5456a14e18883a8acffe844ab9" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45661 } health_report { overall_health: UNKNOWN } } }
I20260811 11:59:30.629771 28863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:30.625106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43500 (local address 127.6.143.131:42265)
0811 11:59:30.625239 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:30.625243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:30.625257 (+ 14us) server_negotiation.cc:407] Connection header received
0811 11:59:30.625337 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:30.625357 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:30.625437 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:30.625599 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 11:59:30.625919 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.626794 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.628085 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:30.628451 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:30.628578 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:30.628928 (+ 350us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 11:59:30.629360 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 11:59:30.629459 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:30.629535 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 11:59:30.629582 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 11:59:30.636029 28661 raft_consensus.cc:1261] T 5fb7352559844e7aa4996ebbfaa12de8 P c06946c9100d40c7a2c4d0390902ac77 [term 1 FOLLOWER]: Refusing update from remote peer 80d90edfd12c42b99570288ea7279f14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:59:30.636040 28522 raft_consensus.cc:1261] T 5fb7352559844e7aa4996ebbfaa12de8 P 208e1a5456a14e18883a8acffe844ab9 [term 1 FOLLOWER]: Refusing update from remote peer 80d90edfd12c42b99570288ea7279f14: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:59:30.636807 28878 consensus_queue.cc:1035] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "208e1a5456a14e18883a8acffe844ab9" member_type: VOTER last_known_addr { host: "127.6.143.129" port: 45661 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:59:30.636997 28876 consensus_queue.cc:1035] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c06946c9100d40c7a2c4d0390902ac77" member_type: VOTER last_known_addr { host: "127.6.143.130" port: 34525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:59:30.647322 28882 mvcc.cc:205] Tried to move back new op lower bound from 7317297441317146624 to 7317297441135525888. Current Snapshot: MvccSnapshot[applied={T|T < 7317297441317146624 or (T in {7317297441317146624})}]
I20260811 11:59:30.652323 28374 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:59:30.652498 28374 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 11:59:30.655093 28374 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7e20095ec04645989208f76b720da428: Sending DeleteTablet for 3 replicas of tablet 5fb7352559844e7aa4996ebbfaa12de8
I20260811 11:59:30.655920 28777 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fb7352559844e7aa4996ebbfaa12de8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:59:30 UTC) from {username='slave'} at 127.0.0.1:43488
I20260811 11:59:30.655889 28638 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fb7352559844e7aa4996ebbfaa12de8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:59:30 UTC) from {username='slave'} at 127.0.0.1:55428
I20260811 11:59:30.655992 28501 tablet_service.cc:1514] Processing DeleteTablet for tablet 5fb7352559844e7aa4996ebbfaa12de8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 11:59:30 UTC) from {username='slave'} at 127.0.0.1:41102
I20260811 11:59:30.656448 28892 tablet_replica.cc:330] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14: stopping tablet replica
I20260811 11:59:30.656651 28893 tablet_replica.cc:330] T 5fb7352559844e7aa4996ebbfaa12de8 P 208e1a5456a14e18883a8acffe844ab9: stopping tablet replica
I20260811 11:59:30.656780 28892 raft_consensus.cc:2229] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14 [term 1 LEADER]: Raft consensus shutting down.
I20260811 11:59:30.656980 28893 raft_consensus.cc:2229] T 5fb7352559844e7aa4996ebbfaa12de8 P 208e1a5456a14e18883a8acffe844ab9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:59:30.657143 28892 raft_consensus.cc:2258] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:59:30.657496 28891 tablet_replica.cc:330] T 5fb7352559844e7aa4996ebbfaa12de8 P c06946c9100d40c7a2c4d0390902ac77: stopping tablet replica
I20260811 11:59:30.657310 28893 raft_consensus.cc:2258] T 5fb7352559844e7aa4996ebbfaa12de8 P 208e1a5456a14e18883a8acffe844ab9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:59:30.657792 28891 raft_consensus.cc:2229] T 5fb7352559844e7aa4996ebbfaa12de8 P c06946c9100d40c7a2c4d0390902ac77 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 11:59:30.658128 28891 raft_consensus.cc:2258] T 5fb7352559844e7aa4996ebbfaa12de8 P c06946c9100d40c7a2c4d0390902ac77 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 11:59:30.658484 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 28436
I20260811 11:59:30.663362 28892 ts_tablet_manager.cc:1905] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:59:30.664161 28891 ts_tablet_manager.cc:1905] T 5fb7352559844e7aa4996ebbfaa12de8 P c06946c9100d40c7a2c4d0390902ac77: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 11:59:30.666436 28892 ts_tablet_manager.cc:1918] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:59:30.666613 28892 log.cc:1196] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-2/wal/wals/5fb7352559844e7aa4996ebbfaa12de8
I20260811 11:59:30.667012 28892 ts_tablet_manager.cc:1939] T 5fb7352559844e7aa4996ebbfaa12de8 P 80d90edfd12c42b99570288ea7279f14: Deleting consensus metadata
I20260811 11:59:30.667601 28891 ts_tablet_manager.cc:1918] T 5fb7352559844e7aa4996ebbfaa12de8 P c06946c9100d40c7a2c4d0390902ac77: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 11:59:30.667711 28891 log.cc:1196] T 5fb7352559844e7aa4996ebbfaa12de8 P c06946c9100d40c7a2c4d0390902ac77: Deleting WAL directory at /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786449319379472-6718-0/minicluster-data/ts-1/wal/wals/5fb7352559844e7aa4996ebbfaa12de8
I20260811 11:59:30.668073 28891 ts_tablet_manager.cc:1939] T 5fb7352559844e7aa4996ebbfaa12de8 P c06946c9100d40c7a2c4d0390902ac77: Deleting consensus metadata
I20260811 11:59:30.668696 28358 catalog_manager.cc:4841] TS 80d90edfd12c42b99570288ea7279f14 (127.6.143.131:42265): tablet 5fb7352559844e7aa4996ebbfaa12de8 (table test-table [id=9267da0fe39b4987ae2cb78094fd5442]) successfully deleted
W20260811 11:59:30.669255 28359 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.143.129:45661 (error 108)
I20260811 11:59:30.669359 28360 catalog_manager.cc:4841] TS c06946c9100d40c7a2c4d0390902ac77 (127.6.143.130:34525): tablet 5fb7352559844e7aa4996ebbfaa12de8 (table test-table [id=9267da0fe39b4987ae2cb78094fd5442]) successfully deleted
I20260811 11:59:30.669523 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 28574
I20260811 11:59:30.670564 28865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:30.670242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.129:45661 (local address 127.0.0.1:41110)
0811 11:59:30.670409 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:30.670479 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.129:45661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260811 11:59:30.671021 28359 catalog_manager.cc:4584] TS 208e1a5456a14e18883a8acffe844ab9 (127.6.143.129:45661): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5fb7352559844e7aa4996ebbfaa12de8: Network error: Client connection negotiation failed: client connection to 127.6.143.129:45661: connect: Connection refused (error 111)
I20260811 11:59:30.679368 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 28712
I20260811 11:59:30.685817 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 28342
2026-08-11T11:59:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1178 ms)
[----------] 6 tests from EncryptionPolicyTest (10781 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:59:30 dist-test-slave-zkpd krb5kdc[28898](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:59:30 dist-test-slave-zkpd krb5kdc[28898](info): set up 2 sockets
Aug 11 11:59:30 dist-test-slave-zkpd krb5kdc[28898](info): commencing operation
krb5kdc: starting...
W20260811 11:59:31.768179 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:59:31 dist-test-slave-zkpd krb5kdc[28898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:59:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:59:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 11:59:31.888258 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44297
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33421
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:31.999135 28911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:31.999410 28911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:31.999480 28911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:32.003432 28911 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:59:32.003520 28911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:32.003558 28911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:32.003615 28911 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:59:32.003657 28911 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:59:32.008833 28911 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33421
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44297
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:32.009845 28911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:32.010838 28911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:32.019331 28922 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:32.019344 28918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:32.019338 28919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:32.019591 28911 server_base.cc:1034] running on GCE node
I20260811 11:59:32.020219 28911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:32.021240 28911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:32.022410 28911 hybrid_clock.cc:648] HybridClock initialized: now 1786449572022375 us; error 38 us; skew 500 ppm
Aug 11 11:59:32 dist-test-slave-zkpd krb5kdc[28898](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449572, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:32.025308 28911 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 11:59:32.026151 28911 webserver.cc:460] Webserver started at http://127.0.0.1:43405/ using document root <none> and password file <none>
I20260811 11:59:32.026751 28911 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:32.026844 28911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:32.027081 28911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:32.029127 28911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "646647ae85ab486daf9a646c0aee66b4"
format_stamp: "Formatted at 2026-08-11 11:59:32 on dist-test-slave-zkpd"
server_key: "e6aef8c137aa457063ba18187bba7905"
server_key_iv: "090cb2360af8b6ec4c2474b5fab058b0"
server_key_version: "encryptionkey@0"
I20260811 11:59:32.029683 28911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "646647ae85ab486daf9a646c0aee66b4"
format_stamp: "Formatted at 2026-08-11 11:59:32 on dist-test-slave-zkpd"
server_key: "e6aef8c137aa457063ba18187bba7905"
server_key_iv: "090cb2360af8b6ec4c2474b5fab058b0"
server_key_version: "encryptionkey@0"
I20260811 11:59:32.033353 28911 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 11:59:32.035737 28929 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:32.036936 28911 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:59:32.037088 28911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "646647ae85ab486daf9a646c0aee66b4"
format_stamp: "Formatted at 2026-08-11 11:59:32 on dist-test-slave-zkpd"
server_key: "e6aef8c137aa457063ba18187bba7905"
server_key_iv: "090cb2360af8b6ec4c2474b5fab058b0"
server_key_version: "encryptionkey@0"
I20260811 11:59:32.037199 28911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:32.060365 28911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:32.063066 28911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:32.063268 28911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:32.070830 28911 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44297
I20260811 11:59:32.070901 28987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44297 every 8 connection(s)
I20260811 11:59:32.071811 28911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:59:32.074667 28988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:32.074705 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 28911
I20260811 11:59:32.074846 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:59:32.075150 6718 external_mini_cluster.cc:1428] Setting key cc84d2eb1d806f5a499032325190532f
I20260811 11:59:32.081982 28988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 646647ae85ab486daf9a646c0aee66b4: Bootstrap starting.
I20260811 11:59:32.084491 28988 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 646647ae85ab486daf9a646c0aee66b4: Neither blocks nor log segments found. Creating new log.
Aug 11 11:59:32 dist-test-slave-zkpd krb5kdc[28898](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449571, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 11:59:32.085423 28988 log.cc:824] T 00000000000000000000000000000000 P 646647ae85ab486daf9a646c0aee66b4: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:32.087270 28988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 646647ae85ab486daf9a646c0aee66b4: No bootstrap required, opened a new log
I20260811 11:59:32.090451 28991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:32.076596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40423 (local address 127.0.0.1:44297)
0811 11:59:32.077025 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:32.077043 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:32.077086 (+ 43us) server_negotiation.cc:407] Connection header received
0811 11:59:32.078176 (+ 1090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:32.078202 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:32.078621 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:32.078990 (+ 369us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:32.079687 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:32.081222 (+ 1535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:32.082097 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:32.082753 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:32.082908 (+ 155us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:32.085775 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:32.085780 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:32.085792 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:32.085818 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:32.087550 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:32.088166 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:32.088170 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:32.088175 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:32.088280 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:32.088641 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:32.088643 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:32.088644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:32.089020 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:32.089171 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:32.089550 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 11:59:32.089775 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":118,"threads_started":1}
I20260811 11:59:32.091331 28988 raft_consensus.cc:348] T 00000000000000000000000000000000 P 646647ae85ab486daf9a646c0aee66b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "646647ae85ab486daf9a646c0aee66b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44297 } }
I20260811 11:59:32.091528 28988 raft_consensus.cc:374] T 00000000000000000000000000000000 P 646647ae85ab486daf9a646c0aee66b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:32.091625 28988 raft_consensus.cc:729] T 00000000000000000000000000000000 P 646647ae85ab486daf9a646c0aee66b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 646647ae85ab486daf9a646c0aee66b4, State: Initialized, Role: FOLLOWER
I20260811 11:59:32.092108 28988 consensus_queue.cc:260] T 00000000000000000000000000000000 P 646647ae85ab486daf9a646c0aee66b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "646647ae85ab486daf9a646c0aee66b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44297 } }
I20260811 11:59:32.092269 28988 raft_consensus.cc:388] T 00000000000000000000000000000000 P 646647ae85ab486daf9a646c0aee66b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:59:32.092370 28988 raft_consensus.cc:482] T 00000000000000000000000000000000 P 646647ae85ab486daf9a646c0aee66b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:59:32.092478 28988 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 646647ae85ab486daf9a646c0aee66b4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:32.093420 28988 raft_consensus.cc:504] T 00000000000000000000000000000000 P 646647ae85ab486daf9a646c0aee66b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "646647ae85ab486daf9a646c0aee66b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44297 } }
I20260811 11:59:32.093504 6718 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:59:32.093806 28988 leader_election.cc:302] T 00000000000000000000000000000000 P 646647ae85ab486daf9a646c0aee66b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 646647ae85ab486daf9a646c0aee66b4; no voters:
I20260811 11:59:32.094129 28988 leader_election.cc:288] T 00000000000000000000000000000000 P 646647ae85ab486daf9a646c0aee66b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:59:32.094194 28993 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 646647ae85ab486daf9a646c0aee66b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:32.094383 28993 raft_consensus.cc:686] T 00000000000000000000000000000000 P 646647ae85ab486daf9a646c0aee66b4 [term 1 LEADER]: Becoming Leader. State: Replica: 646647ae85ab486daf9a646c0aee66b4, State: Running, Role: LEADER
I20260811 11:59:32.094664 28993 consensus_queue.cc:237] T 00000000000000000000000000000000 P 646647ae85ab486daf9a646c0aee66b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "646647ae85ab486daf9a646c0aee66b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44297 } }
I20260811 11:59:32.095368 28988 sys_catalog.cc:564] T 00000000000000000000000000000000 P 646647ae85ab486daf9a646c0aee66b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:59:32.096984 28995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 646647ae85ab486daf9a646c0aee66b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "646647ae85ab486daf9a646c0aee66b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "646647ae85ab486daf9a646c0aee66b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44297 } } }
I20260811 11:59:32.097112 28995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 646647ae85ab486daf9a646c0aee66b4 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:32.097448 28994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 646647ae85ab486daf9a646c0aee66b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 646647ae85ab486daf9a646c0aee66b4. Latest consensus state: current_term: 1 leader_uuid: "646647ae85ab486daf9a646c0aee66b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "646647ae85ab486daf9a646c0aee66b4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44297 } } }
I20260811 11:59:32.097514 28994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 646647ae85ab486daf9a646c0aee66b4 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:32.098380 29008 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:59:32.101464 29008 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:59:32.105484 28991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:32.096223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41535 (local address 127.0.0.1:44297)
0811 11:59:32.096344 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:32.096349 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:32.096364 (+ 15us) server_negotiation.cc:407] Connection header received
0811 11:59:32.096408 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:32.096411 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:32.096449 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:32.096522 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:32.098161 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:32.098942 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:32.099824 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:32.100186 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:32.100291 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:32.101989 (+ 1698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:32.101993 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:32.101995 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:32.102019 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:32.103705 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:32.104228 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:32.104232 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:32.104234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:32.104312 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:32.104641 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:32.104644 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:32.104646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:32.104822 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:32.104974 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:32.105245 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 11:59:32.105338 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 11:59:32.109495 29008 catalog_manager.cc:1347] Generated new cluster ID: 2446472cd52748e890e8052a22f1a2d5
I20260811 11:59:32.109562 29008 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:59:32.120997 29008 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:59:32.121889 29008 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:59:32.128575 29008 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 646647ae85ab486daf9a646c0aee66b4: Generated new TSK 0
I20260811 11:59:32.129230 29008 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:59:32.138854 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 28911
W20260811 11:59:32.150628 28895 connection.cc:535] client connection to 127.0.0.1:44297 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44297): Connection reset by peer (error 104)
2026-08-11T11:59:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1455 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:59:32 dist-test-slave-zkpd krb5kdc[29021](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:59:32 dist-test-slave-zkpd krb5kdc[29021](info): set up 2 sockets
Aug 11 11:59:32 dist-test-slave-zkpd krb5kdc[29021](info): commencing operation
krb5kdc: starting...
W20260811 11:59:33.289179 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:59:33 dist-test-slave-zkpd krb5kdc[29021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449573, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:59:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:59:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 11:59:33.414189 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40429
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38499
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40429
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:33.528448 29035 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:33.528687 29035 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:33.528726 29035 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:33.532572 29035 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:59:33.532620 29035 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:33.532639 29035 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:33.532655 29035 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:59:33.532670 29035 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:59:33.537743 29035 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38499
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40429
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40429
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29035
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:33.538602 29035 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:33.539510 29035 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:33.547685 29041 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:33.547721 29040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:33.547721 29043 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:33.547847 29035 server_base.cc:1034] running on GCE node
I20260811 11:59:33.548413 29035 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:33.549433 29035 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:33.550592 29035 hybrid_clock.cc:648] HybridClock initialized: now 1786449573550561 us; error 41 us; skew 500 ppm
Aug 11 11:59:33 dist-test-slave-zkpd krb5kdc[29021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449573, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:33.553359 29035 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 11:59:33.554194 29035 webserver.cc:460] Webserver started at http://127.0.0.1:34059/ using document root <none> and password file <none>
I20260811 11:59:33.554735 29035 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:33.554829 29035 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:33.555068 29035 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:33.557083 29035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "b4160ce70eb242ada83e1d78639eddd7"
format_stamp: "Formatted at 2026-08-11 11:59:33 on dist-test-slave-zkpd"
server_key: "15a0fc2c74565c947570f5384868e965"
server_key_iv: "cd6cbf8d0ab9078b0c20ebbe4d35e95c"
server_key_version: "encryptionkey@0"
I20260811 11:59:33.557628 29035 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "b4160ce70eb242ada83e1d78639eddd7"
format_stamp: "Formatted at 2026-08-11 11:59:33 on dist-test-slave-zkpd"
server_key: "15a0fc2c74565c947570f5384868e965"
server_key_iv: "cd6cbf8d0ab9078b0c20ebbe4d35e95c"
server_key_version: "encryptionkey@0"
I20260811 11:59:33.561307 29035 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 11:59:33.563561 29052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:33.564782 29035 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:59:33.564941 29035 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "b4160ce70eb242ada83e1d78639eddd7"
format_stamp: "Formatted at 2026-08-11 11:59:33 on dist-test-slave-zkpd"
server_key: "15a0fc2c74565c947570f5384868e965"
server_key_iv: "cd6cbf8d0ab9078b0c20ebbe4d35e95c"
server_key_version: "encryptionkey@0"
I20260811 11:59:33.565062 29035 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:33.582808 29035 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:33.586134 29035 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:33.586336 29035 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:33.593570 29035 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40429
I20260811 11:59:33.593636 29112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40429 every 8 connection(s)
I20260811 11:59:33.594542 29035 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:59:33.597399 29113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:33.600117 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 29035
I20260811 11:59:33.600275 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:59:33.600580 6718 external_mini_cluster.cc:1428] Setting key 3f8ad6065e7c76be5f5adf126242c34f
I20260811 11:59:33.603595 29113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4160ce70eb242ada83e1d78639eddd7: Bootstrap starting.
I20260811 11:59:33.606297 29113 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4160ce70eb242ada83e1d78639eddd7: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:33.607151 29113 log.cc:824] T 00000000000000000000000000000000 P b4160ce70eb242ada83e1d78639eddd7: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:33.609133 29113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4160ce70eb242ada83e1d78639eddd7: No bootstrap required, opened a new log
I20260811 11:59:33.610401 29116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:33.601924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34901 (local address 127.0.0.1:40429)
0811 11:59:33.602368 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:33.602387 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:33.602431 (+ 44us) server_negotiation.cc:407] Connection header received
0811 11:59:33.603229 (+ 798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:33.603252 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:33.603666 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:33.604047 (+ 381us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:33.604466 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:33.605929 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:33.606884 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:33.607597 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:33.607782 (+ 185us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:33.608708 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:33.608713 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:33.608726 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:33.609104 (+ 378us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:33.609159 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:33.609181 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:33.609350 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:33.609545 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 11:59:33.609693 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":122,"threads_started":1}
I20260811 11:59:33.613153 29113 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4160ce70eb242ada83e1d78639eddd7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4160ce70eb242ada83e1d78639eddd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40429 } }
I20260811 11:59:33.613440 29113 raft_consensus.cc:374] T 00000000000000000000000000000000 P b4160ce70eb242ada83e1d78639eddd7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:33.613509 29113 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4160ce70eb242ada83e1d78639eddd7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4160ce70eb242ada83e1d78639eddd7, State: Initialized, Role: FOLLOWER
I20260811 11:59:33.613541 6718 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:59:33.613973 29113 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4160ce70eb242ada83e1d78639eddd7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4160ce70eb242ada83e1d78639eddd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40429 } }
I20260811 11:59:33.614152 29113 raft_consensus.cc:388] T 00000000000000000000000000000000 P b4160ce70eb242ada83e1d78639eddd7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:59:33.614220 29113 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4160ce70eb242ada83e1d78639eddd7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:59:33.614315 29113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4160ce70eb242ada83e1d78639eddd7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:33.615454 29113 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4160ce70eb242ada83e1d78639eddd7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4160ce70eb242ada83e1d78639eddd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40429 } }
I20260811 11:59:33.615859 29113 leader_election.cc:302] T 00000000000000000000000000000000 P b4160ce70eb242ada83e1d78639eddd7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b4160ce70eb242ada83e1d78639eddd7; no voters:
I20260811 11:59:33.616248 29113 leader_election.cc:288] T 00000000000000000000000000000000 P b4160ce70eb242ada83e1d78639eddd7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:59:33.616323 29123 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4160ce70eb242ada83e1d78639eddd7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:33.616551 29123 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4160ce70eb242ada83e1d78639eddd7 [term 1 LEADER]: Becoming Leader. State: Replica: b4160ce70eb242ada83e1d78639eddd7, State: Running, Role: LEADER
I20260811 11:59:33.616946 29123 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4160ce70eb242ada83e1d78639eddd7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4160ce70eb242ada83e1d78639eddd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40429 } }
I20260811 11:59:33.617537 29113 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4160ce70eb242ada83e1d78639eddd7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:59:33.618573 29126 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4160ce70eb242ada83e1d78639eddd7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4160ce70eb242ada83e1d78639eddd7. Latest consensus state: current_term: 1 leader_uuid: "b4160ce70eb242ada83e1d78639eddd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4160ce70eb242ada83e1d78639eddd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40429 } } }
I20260811 11:59:33.618969 29126 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4160ce70eb242ada83e1d78639eddd7 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:33.618863 29125 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4160ce70eb242ada83e1d78639eddd7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b4160ce70eb242ada83e1d78639eddd7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b4160ce70eb242ada83e1d78639eddd7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40429 } } }
I20260811 11:59:33.619321 29125 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4160ce70eb242ada83e1d78639eddd7 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:33.621037 29116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:33.615795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47977 (local address 127.0.0.1:40429)
0811 11:59:33.615949 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:33.615953 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:33.616142 (+ 189us) server_negotiation.cc:407] Connection header received
0811 11:59:33.616304 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:33.616308 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:33.616356 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:33.616446 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:33.616912 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:33.617651 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:33.618892 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:33.619253 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:33.619372 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:33.620427 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:33.620431 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:33.620433 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:33.620448 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:33.620479 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:33.620493 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:33.620583 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:33.620895 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 11:59:33.620932 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 11:59:33.623207 29140 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b4160ce70eb242ada83e1d78639eddd7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 11:59:33.623289 29140 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 11:59:33.623409 29135 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:59:33.624276 29135 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:59:33.629244 29135 catalog_manager.cc:1347] Generated new cluster ID: 62b40ffdb8e9457f8d4a8c155a86d1c9
I20260811 11:59:33.629307 29135 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:59:33.652838 29135 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:59:33.653795 29135 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:59:33.666268 29135 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4160ce70eb242ada83e1d78639eddd7: Generated new TSK 0
I20260811 11:59:33.667135 29135 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:59:33.679031 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 29035
W20260811 11:59:33.687152 29019 connection.cc:535] client connection to 127.0.0.1:40429 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40429): Connection reset by peer (error 104)
2026-08-11T11:59:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1536 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:59:33 dist-test-slave-zkpd krb5kdc[29146](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:59:33 dist-test-slave-zkpd krb5kdc[29146](info): set up 2 sockets
Aug 11 11:59:33 dist-test-slave-zkpd krb5kdc[29146](info): commencing operation
krb5kdc: starting...
W20260811 11:59:34.804476 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.103s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:59:34 dist-test-slave-zkpd krb5kdc[29146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449574, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:59:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:59:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 11:59:34.928156 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41139
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41197
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41139
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:35.039837 29160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:35.040102 29160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:35.040143 29160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:35.044121 29160 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:59:35.044173 29160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:35.044190 29160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:35.044207 29160 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:59:35.044221 29160 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:59:35.049324 29160 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41197
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41139
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:35.050340 29160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:35.051373 29160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:35.059907 29165 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:35.059986 29160 server_base.cc:1034] running on GCE node
W20260811 11:59:35.059978 29166 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:35.059978 29168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:35.060752 29160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:35.061790 29160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:35.062994 29160 hybrid_clock.cc:648] HybridClock initialized: now 1786449575062969 us; error 39 us; skew 500 ppm
Aug 11 11:59:35 dist-test-slave-zkpd krb5kdc[29146](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449575, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:35.065757 29160 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 11:59:35.066596 29160 webserver.cc:460] Webserver started at http://127.0.0.1:40239/ using document root <none> and password file <none>
I20260811 11:59:35.067104 29160 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:35.067160 29160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:35.067332 29160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:35.069351 29160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "9363210523a74dbeb70400cff92d2eaa"
format_stamp: "Formatted at 2026-08-11 11:59:35 on dist-test-slave-zkpd"
server_key: "999ea337c73c8d16b9f9fafd9928ee8e"
server_key_iv: "dfc4d83049a4ff26b4e8c189248a8425"
server_key_version: "encryptionkey@0"
I20260811 11:59:35.069872 29160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "9363210523a74dbeb70400cff92d2eaa"
format_stamp: "Formatted at 2026-08-11 11:59:35 on dist-test-slave-zkpd"
server_key: "999ea337c73c8d16b9f9fafd9928ee8e"
server_key_iv: "dfc4d83049a4ff26b4e8c189248a8425"
server_key_version: "encryptionkey@0"
I20260811 11:59:35.073410 29160 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260811 11:59:35.075752 29175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:35.077026 29160 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 11:59:35.077180 29160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "9363210523a74dbeb70400cff92d2eaa"
format_stamp: "Formatted at 2026-08-11 11:59:35 on dist-test-slave-zkpd"
server_key: "999ea337c73c8d16b9f9fafd9928ee8e"
server_key_iv: "dfc4d83049a4ff26b4e8c189248a8425"
server_key_version: "encryptionkey@0"
I20260811 11:59:35.077284 29160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:35.081914 29160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:35.085100 29160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:35.085291 29160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:35.092465 29160 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41139
I20260811 11:59:35.092516 29232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41139 every 8 connection(s)
I20260811 11:59:35.093313 29160 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:59:35.093650 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 29160
I20260811 11:59:35.093837 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:59:35.094179 6718 external_mini_cluster.cc:1428] Setting key b3b4891ded16a73c93d3d0d7b302c4a4
I20260811 11:59:35.096652 29233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:35.099452 29236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:35.095716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37707 (local address 127.0.0.1:41139)
0811 11:59:35.096174 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:35.096203 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:35.096241 (+ 38us) server_negotiation.cc:407] Connection header received
0811 11:59:35.096924 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:35.096941 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:35.097220 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:35.097486 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:35.098253 (+ 767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:35.098258 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:35.098270 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:35.098280 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:35.098330 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:35.098347 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:35.098457 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:35.098593 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 11:59:35.098759 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":166,"threads_started":1}
I20260811 11:59:35.102337 6718 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:59:35.102984 29233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9363210523a74dbeb70400cff92d2eaa: Bootstrap starting.
I20260811 11:59:35.105535 29233 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9363210523a74dbeb70400cff92d2eaa: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:35.106259 29233 log.cc:824] T 00000000000000000000000000000000 P 9363210523a74dbeb70400cff92d2eaa: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:35.108345 29233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9363210523a74dbeb70400cff92d2eaa: No bootstrap required, opened a new log
I20260811 11:59:35.109864 29236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:35.104451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43875 (local address 127.0.0.1:41139)
0811 11:59:35.104578 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:35.104582 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:35.104762 (+ 180us) server_negotiation.cc:407] Connection header received
0811 11:59:35.104870 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:35.104872 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:35.104899 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:35.104967 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:35.109382 (+ 4415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:35.109384 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:35.109387 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:35.109399 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:35.109432 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:35.109443 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:35.109522 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:35.109643 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 11:59:35.109674 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 11:59:35.112474 29233 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9363210523a74dbeb70400cff92d2eaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9363210523a74dbeb70400cff92d2eaa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41139 } }
I20260811 11:59:35.112692 29233 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9363210523a74dbeb70400cff92d2eaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:35.112768 29233 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9363210523a74dbeb70400cff92d2eaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9363210523a74dbeb70400cff92d2eaa, State: Initialized, Role: FOLLOWER
I20260811 11:59:35.113293 29233 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9363210523a74dbeb70400cff92d2eaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9363210523a74dbeb70400cff92d2eaa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41139 } }
I20260811 11:59:35.113487 29233 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9363210523a74dbeb70400cff92d2eaa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:59:35.113613 29233 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9363210523a74dbeb70400cff92d2eaa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:59:35.113760 29233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9363210523a74dbeb70400cff92d2eaa [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:35.114698 29233 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9363210523a74dbeb70400cff92d2eaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9363210523a74dbeb70400cff92d2eaa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41139 } }
I20260811 11:59:35.114990 29233 leader_election.cc:302] T 00000000000000000000000000000000 P 9363210523a74dbeb70400cff92d2eaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9363210523a74dbeb70400cff92d2eaa; no voters:
I20260811 11:59:35.115232 29233 leader_election.cc:288] T 00000000000000000000000000000000 P 9363210523a74dbeb70400cff92d2eaa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:59:35.115432 29243 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9363210523a74dbeb70400cff92d2eaa [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:35.115725 29243 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9363210523a74dbeb70400cff92d2eaa [term 1 LEADER]: Becoming Leader. State: Replica: 9363210523a74dbeb70400cff92d2eaa, State: Running, Role: LEADER
I20260811 11:59:35.116058 29243 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9363210523a74dbeb70400cff92d2eaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9363210523a74dbeb70400cff92d2eaa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41139 } }
I20260811 11:59:35.116406 29233 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9363210523a74dbeb70400cff92d2eaa [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:59:35.117864 29244 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9363210523a74dbeb70400cff92d2eaa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9363210523a74dbeb70400cff92d2eaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9363210523a74dbeb70400cff92d2eaa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41139 } } }
I20260811 11:59:35.117861 29245 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9363210523a74dbeb70400cff92d2eaa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9363210523a74dbeb70400cff92d2eaa. Latest consensus state: current_term: 1 leader_uuid: "9363210523a74dbeb70400cff92d2eaa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9363210523a74dbeb70400cff92d2eaa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41139 } } }
I20260811 11:59:35.118012 29245 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9363210523a74dbeb70400cff92d2eaa [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:35.118012 29244 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9363210523a74dbeb70400cff92d2eaa [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:35.118921 29253 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:59:35.121425 29253 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:59:35.126497 29253 catalog_manager.cc:1347] Generated new cluster ID: 0c615c5a7a8243bb8f3c685137f97112
I20260811 11:59:35.126561 29253 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:59:35.153005 29253 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:59:35.153326 29253 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:59:35.160601 29253 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9363210523a74dbeb70400cff92d2eaa: Generated new TSK 0
I20260811 11:59:35.161367 29253 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:59:35.168068 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 29160
2026-08-11T11:59:35Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1489 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:59:35 dist-test-slave-zkpd krb5kdc[29266](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:59:35 dist-test-slave-zkpd krb5kdc[29266](info): set up 2 sockets
Aug 11 11:59:35 dist-test-slave-zkpd krb5kdc[29266](info): commencing operation
krb5kdc: starting...
W20260811 11:59:36.261384 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:59:36 dist-test-slave-zkpd krb5kdc[29266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:59:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:59:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 11:59:36.384302 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33065
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:36.494937 29280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:36.495169 29280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:36.495208 29280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:36.499193 29280 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:59:36.499248 29280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:36.499267 29280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:36.499284 29280 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:59:36.499300 29280 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:59:36.504347 29280 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:36.505759 29280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:36.506798 29280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:36.515229 29288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:36.515182 29285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:36.515192 29286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:36.515435 29280 server_base.cc:1034] running on GCE node
I20260811 11:59:36.516109 29280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:36.517112 29280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:36.518275 29280 hybrid_clock.cc:648] HybridClock initialized: now 1786449576518245 us; error 40 us; skew 500 ppm
Aug 11 11:59:36 dist-test-slave-zkpd krb5kdc[29266](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449576, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:36.521023 29280 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 11:59:36.521864 29280 webserver.cc:460] Webserver started at http://127.0.0.1:46519/ using document root <none> and password file <none>
I20260811 11:59:36.522409 29280 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:36.522469 29280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:36.522704 29280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:36.524778 29280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "3253f1d5e668423e81bf66a50bcf5992"
format_stamp: "Formatted at 2026-08-11 11:59:36 on dist-test-slave-zkpd"
server_key: "0d187a275262b676362a0e4f1ea751c7"
server_key_iv: "b7a8289231d24c188bb50c5d9663adf9"
server_key_version: "encryptionkey@0"
I20260811 11:59:36.525332 29280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "3253f1d5e668423e81bf66a50bcf5992"
format_stamp: "Formatted at 2026-08-11 11:59:36 on dist-test-slave-zkpd"
server_key: "0d187a275262b676362a0e4f1ea751c7"
server_key_iv: "b7a8289231d24c188bb50c5d9663adf9"
server_key_version: "encryptionkey@0"
I20260811 11:59:36.529033 29280 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 11:59:36.531365 29296 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:36.532634 29280 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:59:36.532781 29280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "3253f1d5e668423e81bf66a50bcf5992"
format_stamp: "Formatted at 2026-08-11 11:59:36 on dist-test-slave-zkpd"
server_key: "0d187a275262b676362a0e4f1ea751c7"
server_key_iv: "b7a8289231d24c188bb50c5d9663adf9"
server_key_version: "encryptionkey@0"
I20260811 11:59:36.532892 29280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:36.550895 29280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:36.553725 29280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:36.553928 29280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:36.561169 29280 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33065
I20260811 11:59:36.561194 29353 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33065 every 8 connection(s)
I20260811 11:59:36.562126 29280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:59:36.564858 29354 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:36.570333 29354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3253f1d5e668423e81bf66a50bcf5992: Bootstrap starting.
I20260811 11:59:36.570612 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 29280
I20260811 11:59:36.570756 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:59:36.571074 6718 external_mini_cluster.cc:1428] Setting key 2732500d78489c5c1c002465348d7bed
I20260811 11:59:36.573071 29354 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3253f1d5e668423e81bf66a50bcf5992: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:36.573813 29354 log.cc:824] T 00000000000000000000000000000000 P 3253f1d5e668423e81bf66a50bcf5992: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:36.576012 29354 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3253f1d5e668423e81bf66a50bcf5992: No bootstrap required, opened a new log
Aug 11 11:59:36 dist-test-slave-zkpd krb5kdc[29266](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449576, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 11:59:36.580650 29354 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3253f1d5e668423e81bf66a50bcf5992 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3253f1d5e668423e81bf66a50bcf5992" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33065 } }
I20260811 11:59:36.580860 29354 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3253f1d5e668423e81bf66a50bcf5992 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:36.580924 29354 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3253f1d5e668423e81bf66a50bcf5992 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3253f1d5e668423e81bf66a50bcf5992, State: Initialized, Role: FOLLOWER
I20260811 11:59:36.581395 29354 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3253f1d5e668423e81bf66a50bcf5992 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3253f1d5e668423e81bf66a50bcf5992" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33065 } }
I20260811 11:59:36.581549 29354 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3253f1d5e668423e81bf66a50bcf5992 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:59:36.581681 29354 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3253f1d5e668423e81bf66a50bcf5992 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:59:36.581802 29354 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3253f1d5e668423e81bf66a50bcf5992 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:36.582636 29354 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3253f1d5e668423e81bf66a50bcf5992 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3253f1d5e668423e81bf66a50bcf5992" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33065 } }
I20260811 11:59:36.583014 29354 leader_election.cc:302] T 00000000000000000000000000000000 P 3253f1d5e668423e81bf66a50bcf5992 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3253f1d5e668423e81bf66a50bcf5992; no voters:
I20260811 11:59:36.583343 29354 leader_election.cc:288] T 00000000000000000000000000000000 P 3253f1d5e668423e81bf66a50bcf5992 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:59:36.583412 29361 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3253f1d5e668423e81bf66a50bcf5992 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:36.583635 29361 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3253f1d5e668423e81bf66a50bcf5992 [term 1 LEADER]: Becoming Leader. State: Replica: 3253f1d5e668423e81bf66a50bcf5992, State: Running, Role: LEADER
I20260811 11:59:36.584002 29361 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3253f1d5e668423e81bf66a50bcf5992 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3253f1d5e668423e81bf66a50bcf5992" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33065 } }
I20260811 11:59:36.584626 29354 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3253f1d5e668423e81bf66a50bcf5992 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:59:36.585747 29362 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3253f1d5e668423e81bf66a50bcf5992 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3253f1d5e668423e81bf66a50bcf5992" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3253f1d5e668423e81bf66a50bcf5992" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33065 } } }
I20260811 11:59:36.585876 29362 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3253f1d5e668423e81bf66a50bcf5992 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:36.586196 29363 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3253f1d5e668423e81bf66a50bcf5992 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3253f1d5e668423e81bf66a50bcf5992. Latest consensus state: current_term: 1 leader_uuid: "3253f1d5e668423e81bf66a50bcf5992" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3253f1d5e668423e81bf66a50bcf5992" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33065 } } }
I20260811 11:59:36.586294 29363 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3253f1d5e668423e81bf66a50bcf5992 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:36.586279 29370 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:59:36.586611 29358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:36.572435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35961 (local address 127.0.0.1:33065)
0811 11:59:36.572880 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:36.572895 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:36.572929 (+ 34us) server_negotiation.cc:407] Connection header received
0811 11:59:36.573591 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:36.573612 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:36.574012 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:36.574363 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:36.575234 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:36.576401 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:36.577580 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:36.578032 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:36.578149 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:36.581081 (+ 2932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:36.581086 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:36.581099 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:36.581126 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:36.583158 (+ 2032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:36.583702 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:36.583708 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:36.583714 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:36.583847 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:36.584198 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:36.584201 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:36.584218 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:36.584579 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:36.584795 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:36.585234 (+ 439us) server_negotiation.cc:299] Negotiation successful
0811 11:59:36.585551 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":127,"threads_started":1}
I20260811 11:59:36.589671 29370 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:59:36.590032 6718 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:59:36.595412 29370 catalog_manager.cc:1347] Generated new cluster ID: 4459416e43d4492593d8dfe4fbe2faba
I20260811 11:59:36.595484 29370 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:59:36.604905 29358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:36.592287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58443 (local address 127.0.0.1:33065)
0811 11:59:36.592401 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:36.592405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:36.596420 (+ 4015us) server_negotiation.cc:407] Connection header received
0811 11:59:36.596545 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:36.596548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:36.596591 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:36.596669 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:36.597486 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:36.598236 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:36.599423 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:36.599801 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:36.599928 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:36.601531 (+ 1603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:36.601535 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:36.601537 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:36.601561 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:36.603192 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:36.603671 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:36.603674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:36.603676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:36.603764 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:36.604075 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:36.604077 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:36.604079 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:36.604276 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:36.604405 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:36.604652 (+ 247us) server_negotiation.cc:299] Negotiation successful
0811 11:59:36.604764 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 11:59:36.635583 29370 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:59:36.636690 29370 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:59:36.643146 29370 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3253f1d5e668423e81bf66a50bcf5992: Generated new TSK 0
I20260811 11:59:36.643864 29370 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:59:36.656836 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 29280
2026-08-11T11:59:36Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1489 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:59:36 dist-test-slave-zkpd krb5kdc[29388](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:59:36 dist-test-slave-zkpd krb5kdc[29388](info): set up 2 sockets
Aug 11 11:59:36 dist-test-slave-zkpd krb5kdc[29388](info): commencing operation
krb5kdc: starting...
W20260811 11:59:37.746053 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:59:37 dist-test-slave-zkpd krb5kdc[29388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449577, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:59:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:59:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 11:59:37.868988 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41349
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:37.980695 29402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:37.980942 29402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:37.981014 29402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:37.984951 29402 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:59:37.985001 29402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:37.985020 29402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:37.985057 29402 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:59:37.985092 29402 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:59:37.990124 29402 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:37.991132 29402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:37.992197 29402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:38.000247 29407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:38.000265 29408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:38.000288 29410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:38.000638 29402 server_base.cc:1034] running on GCE node
I20260811 11:59:38.001157 29402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:38.002137 29402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:38.003317 29402 hybrid_clock.cc:648] HybridClock initialized: now 1786449578003300 us; error 29 us; skew 500 ppm
Aug 11 11:59:38 dist-test-slave-zkpd krb5kdc[29388](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449578, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:38.006155 29402 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 11:59:38.007031 29402 webserver.cc:460] Webserver started at http://127.0.0.1:34535/ using document root <none> and password file <none>
I20260811 11:59:38.007596 29402 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:38.007690 29402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:38.007997 29402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:38.010097 29402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "40c1f2b18653411db616721f3c3696ec"
format_stamp: "Formatted at 2026-08-11 11:59:38 on dist-test-slave-zkpd"
server_key: "ec213df36db9d0d801698c898340eaec"
server_key_iv: "9ce3209f07cc8477a1f953d63c501bc8"
server_key_version: "encryptionkey@0"
I20260811 11:59:38.010682 29402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "40c1f2b18653411db616721f3c3696ec"
format_stamp: "Formatted at 2026-08-11 11:59:38 on dist-test-slave-zkpd"
server_key: "ec213df36db9d0d801698c898340eaec"
server_key_iv: "9ce3209f07cc8477a1f953d63c501bc8"
server_key_version: "encryptionkey@0"
I20260811 11:59:38.014474 29402 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 11:59:38.016902 29418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:38.018077 29402 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:59:38.018217 29402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "40c1f2b18653411db616721f3c3696ec"
format_stamp: "Formatted at 2026-08-11 11:59:38 on dist-test-slave-zkpd"
server_key: "ec213df36db9d0d801698c898340eaec"
server_key_iv: "9ce3209f07cc8477a1f953d63c501bc8"
server_key_version: "encryptionkey@0"
I20260811 11:59:38.018325 29402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:38.033337 29402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:38.036401 29402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:38.036618 29402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:38.043895 29402 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41349
I20260811 11:59:38.043951 29475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41349 every 8 connection(s)
I20260811 11:59:38.044920 29402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:59:38.047689 29476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:38.053323 29476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40c1f2b18653411db616721f3c3696ec: Bootstrap starting.
I20260811 11:59:38.054978 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 29402
I20260811 11:59:38.055130 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:59:38.055433 6718 external_mini_cluster.cc:1428] Setting key c60b17d94793faf22b43a6a3a96ac0c6
I20260811 11:59:38.055960 29476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 40c1f2b18653411db616721f3c3696ec: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:38.056893 29476 log.cc:824] T 00000000000000000000000000000000 P 40c1f2b18653411db616721f3c3696ec: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:38.058903 29476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 40c1f2b18653411db616721f3c3696ec: No bootstrap required, opened a new log
I20260811 11:59:38.062346 29476 raft_consensus.cc:348] T 00000000000000000000000000000000 P 40c1f2b18653411db616721f3c3696ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c1f2b18653411db616721f3c3696ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41349 } }
I20260811 11:59:38.062588 29476 raft_consensus.cc:374] T 00000000000000000000000000000000 P 40c1f2b18653411db616721f3c3696ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:38.062674 29476 raft_consensus.cc:729] T 00000000000000000000000000000000 P 40c1f2b18653411db616721f3c3696ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40c1f2b18653411db616721f3c3696ec, State: Initialized, Role: FOLLOWER
I20260811 11:59:38.063187 29476 consensus_queue.cc:260] T 00000000000000000000000000000000 P 40c1f2b18653411db616721f3c3696ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c1f2b18653411db616721f3c3696ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41349 } }
I20260811 11:59:38.063342 29476 raft_consensus.cc:388] T 00000000000000000000000000000000 P 40c1f2b18653411db616721f3c3696ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:59:38.063417 29476 raft_consensus.cc:482] T 00000000000000000000000000000000 P 40c1f2b18653411db616721f3c3696ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:59:38.063529 29476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 40c1f2b18653411db616721f3c3696ec [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:38.064468 29479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:38.056604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44975 (local address 127.0.0.1:41349)
0811 11:59:38.057114 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:38.057131 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:38.057168 (+ 37us) server_negotiation.cc:407] Connection header received
0811 11:59:38.057996 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:38.058017 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:38.058357 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:38.058701 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:38.059177 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:38.060208 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:38.061095 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:38.061577 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:38.061751 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:38.062630 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:38.062636 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:38.062654 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:38.063111 (+ 457us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:38.063178 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:38.063202 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:38.063317 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:38.063434 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 11:59:38.063653 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":154,"threads_started":1}
I20260811 11:59:38.064483 29476 raft_consensus.cc:504] T 00000000000000000000000000000000 P 40c1f2b18653411db616721f3c3696ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c1f2b18653411db616721f3c3696ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41349 } }
I20260811 11:59:38.064903 29476 leader_election.cc:302] T 00000000000000000000000000000000 P 40c1f2b18653411db616721f3c3696ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 40c1f2b18653411db616721f3c3696ec; no voters:
I20260811 11:59:38.065243 29476 leader_election.cc:288] T 00000000000000000000000000000000 P 40c1f2b18653411db616721f3c3696ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:59:38.065336 29481 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 40c1f2b18653411db616721f3c3696ec [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:38.065598 29481 raft_consensus.cc:686] T 00000000000000000000000000000000 P 40c1f2b18653411db616721f3c3696ec [term 1 LEADER]: Becoming Leader. State: Replica: 40c1f2b18653411db616721f3c3696ec, State: Running, Role: LEADER
I20260811 11:59:38.065944 29481 consensus_queue.cc:237] T 00000000000000000000000000000000 P 40c1f2b18653411db616721f3c3696ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c1f2b18653411db616721f3c3696ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41349 } }
I20260811 11:59:38.066547 29476 sys_catalog.cc:564] T 00000000000000000000000000000000 P 40c1f2b18653411db616721f3c3696ec [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:59:38.067188 29483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40c1f2b18653411db616721f3c3696ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 40c1f2b18653411db616721f3c3696ec. Latest consensus state: current_term: 1 leader_uuid: "40c1f2b18653411db616721f3c3696ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c1f2b18653411db616721f3c3696ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41349 } } }
I20260811 11:59:38.067792 6718 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:59:38.067953 29483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40c1f2b18653411db616721f3c3696ec [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:38.067863 29482 sys_catalog.cc:455] T 00000000000000000000000000000000 P 40c1f2b18653411db616721f3c3696ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "40c1f2b18653411db616721f3c3696ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "40c1f2b18653411db616721f3c3696ec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41349 } } }
I20260811 11:59:38.068354 29482 sys_catalog.cc:458] T 00000000000000000000000000000000 P 40c1f2b18653411db616721f3c3696ec [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:38.068395 29491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:59:38.071949 29491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:59:38.074172 29479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:38.070147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48071 (local address 127.0.0.1:41349)
0811 11:59:38.070242 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:38.070246 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:38.070542 (+ 296us) server_negotiation.cc:407] Connection header received
0811 11:59:38.070660 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:38.070664 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:38.070712 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:38.070808 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:38.071073 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:38.071568 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:38.072514 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:38.072739 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:38.072852 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:38.073696 (+ 844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:38.073700 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:38.073705 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:38.073720 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:38.073745 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:38.073754 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:38.073834 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:38.074000 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 11:59:38.074053 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 11:59:38.077847 29491 catalog_manager.cc:1347] Generated new cluster ID: 8b7ad9f69f764f77934a7d743d4fd19d
I20260811 11:59:38.077924 29491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:59:38.088622 29491 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:59:38.089484 29491 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:59:38.095750 29491 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 40c1f2b18653411db616721f3c3696ec: Generated new TSK 0
I20260811 11:59:38.096563 29491 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:59:38.109088 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 29402
2026-08-11T11:59:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1451 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:59:38 dist-test-slave-zkpd krb5kdc[29509](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:59:38 dist-test-slave-zkpd krb5kdc[29509](info): set up 2 sockets
Aug 11 11:59:38 dist-test-slave-zkpd krb5kdc[29509](info): commencing operation
krb5kdc: starting...
W20260811 11:59:39.192548 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:59:39 dist-test-slave-zkpd krb5kdc[29509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449579, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:59:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:59:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 11:59:39.320369 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39703
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39703
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:39.430925 29522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:39.431149 29522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:39.431222 29522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:39.435328 29522 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:59:39.435379 29522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:39.435397 29522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:39.435443 29522 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:59:39.435477 29522 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:59:39.440532 29522 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:39.441488 29522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:39.442416 29522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:39.450893 29527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:39.450902 29530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:39.450898 29528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:39.451251 29522 server_base.cc:1034] running on GCE node
I20260811 11:59:39.451766 29522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:39.452791 29522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:39.453953 29522 hybrid_clock.cc:648] HybridClock initialized: now 1786449579453925 us; error 40 us; skew 500 ppm
Aug 11 11:59:39 dist-test-slave-zkpd krb5kdc[29509](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449579, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:39.456629 29522 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 11:59:39.457474 29522 webserver.cc:460] Webserver started at http://127.0.0.1:45897/ using document root <none> and password file <none>
I20260811 11:59:39.458029 29522 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:39.458110 29522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:39.458385 29522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:39.461292 29522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "6fed8fd213d44058a80f6e69df4bbc11"
format_stamp: "Formatted at 2026-08-11 11:59:39 on dist-test-slave-zkpd"
server_key: "5f2c910381185ec654605f53d1d382c0"
server_key_iv: "91f4302411a9b154bb56f4be33b69f58"
server_key_version: "encryptionkey@0"
I20260811 11:59:39.462018 29522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "6fed8fd213d44058a80f6e69df4bbc11"
format_stamp: "Formatted at 2026-08-11 11:59:39 on dist-test-slave-zkpd"
server_key: "5f2c910381185ec654605f53d1d382c0"
server_key_iv: "91f4302411a9b154bb56f4be33b69f58"
server_key_version: "encryptionkey@0"
I20260811 11:59:39.466723 29522 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 11:59:39.469164 29537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:39.470454 29522 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 11:59:39.470611 29522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "6fed8fd213d44058a80f6e69df4bbc11"
format_stamp: "Formatted at 2026-08-11 11:59:39 on dist-test-slave-zkpd"
server_key: "5f2c910381185ec654605f53d1d382c0"
server_key_iv: "91f4302411a9b154bb56f4be33b69f58"
server_key_version: "encryptionkey@0"
I20260811 11:59:39.470722 29522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:39.475379 29522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:39.478261 29522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:39.478483 29522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:39.485735 29522 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39703
I20260811 11:59:39.485764 29595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39703 every 8 connection(s)
I20260811 11:59:39.486666 29522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:59:39.489485 29596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:39.495019 29596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fed8fd213d44058a80f6e69df4bbc11: Bootstrap starting.
I20260811 11:59:39.496201 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 29522
I20260811 11:59:39.496356 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:59:39.496637 6718 external_mini_cluster.cc:1428] Setting key 7506bb29ab3274ec7e4a7579fbf9a8ea
I20260811 11:59:39.497822 29596 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6fed8fd213d44058a80f6e69df4bbc11: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:39.498934 29596 log.cc:824] T 00000000000000000000000000000000 P 6fed8fd213d44058a80f6e69df4bbc11: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:39.501152 29596 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fed8fd213d44058a80f6e69df4bbc11: No bootstrap required, opened a new log
I20260811 11:59:39.502053 29599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:39.497735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60017 (local address 127.0.0.1:39703)
0811 11:59:39.498282 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:39.498304 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:39.498354 (+ 50us) server_negotiation.cc:407] Connection header received
0811 11:59:39.499265 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:39.499294 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:39.499591 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:39.499973 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:39.500711 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:39.500716 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:39.500731 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:39.500743 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:39.500811 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:39.500842 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:39.500963 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:39.501036 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 11:59:39.501178 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":138,"threads_started":1}
I20260811 11:59:39.504762 6718 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:59:39.505015 29596 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6fed8fd213d44058a80f6e69df4bbc11 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fed8fd213d44058a80f6e69df4bbc11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39703 } }
I20260811 11:59:39.505285 29596 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6fed8fd213d44058a80f6e69df4bbc11 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:39.505347 29596 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6fed8fd213d44058a80f6e69df4bbc11 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fed8fd213d44058a80f6e69df4bbc11, State: Initialized, Role: FOLLOWER
I20260811 11:59:39.505841 29596 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6fed8fd213d44058a80f6e69df4bbc11 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fed8fd213d44058a80f6e69df4bbc11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39703 } }
I20260811 11:59:39.506023 29596 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6fed8fd213d44058a80f6e69df4bbc11 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:59:39.506111 29596 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6fed8fd213d44058a80f6e69df4bbc11 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:59:39.506254 29596 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6fed8fd213d44058a80f6e69df4bbc11 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:39.507369 29596 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6fed8fd213d44058a80f6e69df4bbc11 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fed8fd213d44058a80f6e69df4bbc11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39703 } }
I20260811 11:59:39.507740 29596 leader_election.cc:302] T 00000000000000000000000000000000 P 6fed8fd213d44058a80f6e69df4bbc11 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6fed8fd213d44058a80f6e69df4bbc11; no voters:
I20260811 11:59:39.508091 29596 leader_election.cc:288] T 00000000000000000000000000000000 P 6fed8fd213d44058a80f6e69df4bbc11 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:59:39.508423 29606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6fed8fd213d44058a80f6e69df4bbc11 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:39.508626 29606 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6fed8fd213d44058a80f6e69df4bbc11 [term 1 LEADER]: Becoming Leader. State: Replica: 6fed8fd213d44058a80f6e69df4bbc11, State: Running, Role: LEADER
I20260811 11:59:39.508906 29599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:39.507103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58431 (local address 127.0.0.1:39703)
0811 11:59:39.507241 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:39.507247 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:39.507368 (+ 121us) server_negotiation.cc:407] Connection header received
0811 11:59:39.507437 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:39.507440 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:39.507475 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:39.507563 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:39.508309 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:39.508316 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:39.508322 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:39.508338 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:39.508372 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:39.508384 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:39.508737 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:39.508771 (+ 34us) server_negotiation.cc:299] Negotiation successful
0811 11:59:39.508805 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 11:59:39.508940 29606 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6fed8fd213d44058a80f6e69df4bbc11 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fed8fd213d44058a80f6e69df4bbc11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39703 } }
I20260811 11:59:39.509413 29596 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6fed8fd213d44058a80f6e69df4bbc11 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:59:39.510802 29607 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fed8fd213d44058a80f6e69df4bbc11 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6fed8fd213d44058a80f6e69df4bbc11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fed8fd213d44058a80f6e69df4bbc11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39703 } } }
I20260811 11:59:39.510927 29607 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fed8fd213d44058a80f6e69df4bbc11 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:39.510883 29608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fed8fd213d44058a80f6e69df4bbc11 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6fed8fd213d44058a80f6e69df4bbc11. Latest consensus state: current_term: 1 leader_uuid: "6fed8fd213d44058a80f6e69df4bbc11" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fed8fd213d44058a80f6e69df4bbc11" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39703 } } }
I20260811 11:59:39.511006 29608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fed8fd213d44058a80f6e69df4bbc11 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:39.511374 29615 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:59:39.514415 29615 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:59:39.519562 29615 catalog_manager.cc:1347] Generated new cluster ID: 365fd30c3bf44fb1827c0b53d4fd9169
I20260811 11:59:39.519629 29615 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:59:39.546437 29615 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:59:39.546783 29615 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:59:39.553085 29615 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6fed8fd213d44058a80f6e69df4bbc11: Generated new TSK 0
I20260811 11:59:39.553957 29615 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:59:39.563326 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 29522
2026-08-11T11:59:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1454 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:59:39 dist-test-slave-zkpd krb5kdc[29628](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:59:39 dist-test-slave-zkpd krb5kdc[29628](info): set up 2 sockets
Aug 11 11:59:39 dist-test-slave-zkpd krb5kdc[29628](info): commencing operation
krb5kdc: starting...
W20260811 11:59:40.661543 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.074s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:59:40 dist-test-slave-zkpd krb5kdc[29628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:59:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:59:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 11:59:40.789120 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46531
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46531
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:40.900614 29643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:40.900884 29643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:40.900951 29643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:40.904980 29643 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:59:40.905056 29643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:40.905095 29643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:40.905153 29643 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:59:40.905192 29643 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:59:40.910218 29643 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46531
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46531
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29643
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:40.911157 29643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:40.912164 29643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:40.920501 29652 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:40.920528 29648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:40.920557 29650 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:40.920617 29643 server_base.cc:1034] running on GCE node
I20260811 11:59:40.921268 29643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:40.922272 29643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:40.923460 29643 hybrid_clock.cc:648] HybridClock initialized: now 1786449580923445 us; error 26 us; skew 500 ppm
Aug 11 11:59:40 dist-test-slave-zkpd krb5kdc[29628](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449580, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:40.926249 29643 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 11:59:40.927014 29643 webserver.cc:460] Webserver started at http://127.0.0.1:42251/ using document root <none> and password file <none>
I20260811 11:59:40.927532 29643 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:40.927589 29643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:40.927740 29643 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:40.929682 29643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "e17606e3436b4837bd076da618c64f09"
format_stamp: "Formatted at 2026-08-11 11:59:40 on dist-test-slave-zkpd"
server_key: "e53286b3951e56ce2d5e39f80ed0e385"
server_key_iv: "9c53186e719737786032aca2fe675d1a"
server_key_version: "encryptionkey@0"
I20260811 11:59:40.930215 29643 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "e17606e3436b4837bd076da618c64f09"
format_stamp: "Formatted at 2026-08-11 11:59:40 on dist-test-slave-zkpd"
server_key: "e53286b3951e56ce2d5e39f80ed0e385"
server_key_iv: "9c53186e719737786032aca2fe675d1a"
server_key_version: "encryptionkey@0"
I20260811 11:59:40.933769 29643 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 11:59:40.936129 29659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:40.937320 29643 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:59:40.937443 29643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "e17606e3436b4837bd076da618c64f09"
format_stamp: "Formatted at 2026-08-11 11:59:40 on dist-test-slave-zkpd"
server_key: "e53286b3951e56ce2d5e39f80ed0e385"
server_key_iv: "9c53186e719737786032aca2fe675d1a"
server_key_version: "encryptionkey@0"
I20260811 11:59:40.937548 29643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:40.961337 29643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:40.964686 29643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:40.964905 29643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:40.972007 29643 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46531
I20260811 11:59:40.972077 29714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46531 every 8 connection(s)
I20260811 11:59:40.972926 29643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:59:40.975538 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 29643
I20260811 11:59:40.975713 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:59:40.975991 29715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:40.976078 6718 external_mini_cluster.cc:1428] Setting key cf18ac99bf347ce4077413d224fac9af
I20260811 11:59:40.983706 29715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e17606e3436b4837bd076da618c64f09: Bootstrap starting.
Aug 11 11:59:40 dist-test-slave-zkpd krb5kdc[29628](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449580, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 11:59:40.986366 29715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e17606e3436b4837bd076da618c64f09: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:40.987243 29715 log.cc:824] T 00000000000000000000000000000000 P e17606e3436b4837bd076da618c64f09: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:40.989459 29715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e17606e3436b4837bd076da618c64f09: No bootstrap required, opened a new log
I20260811 11:59:40.990736 29718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:40.977373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:38291 (local address 127.0.0.1:46531)
0811 11:59:40.977770 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:40.977794 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:40.977829 (+ 35us) server_negotiation.cc:407] Connection header received
0811 11:59:40.978679 (+ 850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:40.978698 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:40.979028 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:40.979303 (+ 275us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:40.980318 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:40.981298 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:40.982526 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:40.982971 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:40.983085 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:40.985846 (+ 2761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:40.985851 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:40.985877 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:40.985903 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:40.987866 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:40.988475 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:40.988480 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:40.988484 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:40.988590 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:40.988944 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:40.988946 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:40.988947 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:40.989297 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:40.989467 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:40.989792 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 11:59:40.990021 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":127,"threads_started":1}
I20260811 11:59:40.993790 6718 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:59:40.993620 29715 raft_consensus.cc:348] T 00000000000000000000000000000000 P e17606e3436b4837bd076da618c64f09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e17606e3436b4837bd076da618c64f09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46531 } }
I20260811 11:59:40.993937 29715 raft_consensus.cc:374] T 00000000000000000000000000000000 P e17606e3436b4837bd076da618c64f09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:40.993993 29715 raft_consensus.cc:729] T 00000000000000000000000000000000 P e17606e3436b4837bd076da618c64f09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e17606e3436b4837bd076da618c64f09, State: Initialized, Role: FOLLOWER
I20260811 11:59:40.994455 29715 consensus_queue.cc:260] T 00000000000000000000000000000000 P e17606e3436b4837bd076da618c64f09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e17606e3436b4837bd076da618c64f09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46531 } }
I20260811 11:59:40.994637 29715 raft_consensus.cc:388] T 00000000000000000000000000000000 P e17606e3436b4837bd076da618c64f09 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:59:40.994728 29715 raft_consensus.cc:482] T 00000000000000000000000000000000 P e17606e3436b4837bd076da618c64f09 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:59:40.994834 29715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e17606e3436b4837bd076da618c64f09 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:40.996115 29715 raft_consensus.cc:504] T 00000000000000000000000000000000 P e17606e3436b4837bd076da618c64f09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e17606e3436b4837bd076da618c64f09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46531 } }
I20260811 11:59:40.996508 29715 leader_election.cc:302] T 00000000000000000000000000000000 P e17606e3436b4837bd076da618c64f09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e17606e3436b4837bd076da618c64f09; no voters:
I20260811 11:59:40.996882 29715 leader_election.cc:288] T 00000000000000000000000000000000 P e17606e3436b4837bd076da618c64f09 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:59:40.996994 29725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e17606e3436b4837bd076da618c64f09 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:40.997242 29725 raft_consensus.cc:686] T 00000000000000000000000000000000 P e17606e3436b4837bd076da618c64f09 [term 1 LEADER]: Becoming Leader. State: Replica: e17606e3436b4837bd076da618c64f09, State: Running, Role: LEADER
I20260811 11:59:40.997575 29725 consensus_queue.cc:237] T 00000000000000000000000000000000 P e17606e3436b4837bd076da618c64f09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e17606e3436b4837bd076da618c64f09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46531 } }
I20260811 11:59:40.998240 29715 sys_catalog.cc:564] T 00000000000000000000000000000000 P e17606e3436b4837bd076da618c64f09 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:59:40.999986 29727 sys_catalog.cc:455] T 00000000000000000000000000000000 P e17606e3436b4837bd076da618c64f09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e17606e3436b4837bd076da618c64f09. Latest consensus state: current_term: 1 leader_uuid: "e17606e3436b4837bd076da618c64f09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e17606e3436b4837bd076da618c64f09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46531 } } }
I20260811 11:59:41.000128 29727 sys_catalog.cc:458] T 00000000000000000000000000000000 P e17606e3436b4837bd076da618c64f09 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:41.000269 29726 sys_catalog.cc:455] T 00000000000000000000000000000000 P e17606e3436b4837bd076da618c64f09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e17606e3436b4837bd076da618c64f09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e17606e3436b4837bd076da618c64f09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46531 } } }
I20260811 11:59:41.000377 29726 sys_catalog.cc:458] T 00000000000000000000000000000000 P e17606e3436b4837bd076da618c64f09 [sys.catalog]: This master's current role is: LEADER
W20260811 11:59:41.003244 29742 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e17606e3436b4837bd076da618c64f09: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 11:59:41.003333 29742 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 11:59:41.004000 29743 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:59:41.004964 29743 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:59:41.005698 29718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:40.996159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:46391 (local address 127.0.0.1:46531)
0811 11:59:40.996289 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:40.996293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:40.996451 (+ 158us) server_negotiation.cc:407] Connection header received
0811 11:59:40.996550 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:40.996554 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:40.996601 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:40.996688 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:40.997531 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:40.998283 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:40.999330 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:40.999670 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:40.999781 (+ 111us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:41.001793 (+ 2012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:41.001797 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:41.001799 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:41.001824 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:41.003629 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:41.004198 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:41.004201 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:41.004203 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:41.004291 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:41.004714 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:41.004717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:41.004719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:41.004941 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:41.005107 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:41.005446 (+ 339us) server_negotiation.cc:299] Negotiation successful
0811 11:59:41.005546 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 11:59:41.010366 29743 catalog_manager.cc:1347] Generated new cluster ID: 5d71b1df51cb49d481c4a4dcb0864f31
I20260811 11:59:41.010433 29743 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:59:41.029726 29743 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:59:41.030601 29743 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:59:41.047324 29743 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e17606e3436b4837bd076da618c64f09: Generated new TSK 0
I20260811 11:59:41.048082 29743 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:59:41.056392 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 29643
W20260811 11:59:41.066295 29625 connection.cc:535] client connection to 127.0.0.1:46531 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46531): Connection reset by peer (error 104)
2026-08-11T11:59:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1493 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:59:41 dist-test-slave-zkpd krb5kdc[29750](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:59:41 dist-test-slave-zkpd krb5kdc[29750](info): set up 2 sockets
Aug 11 11:59:41 dist-test-slave-zkpd krb5kdc[29750](info): commencing operation
krb5kdc: starting...
W20260811 11:59:42.137384 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:59:42 dist-test-slave-zkpd krb5kdc[29750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449582, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:59:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:59:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 11:59:42.262754 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43535
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43535
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:42.377907 29763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:42.378156 29763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:42.378198 29763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:42.382335 29763 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:59:42.382390 29763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:42.382409 29763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:42.382426 29763 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:59:42.382440 29763 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:59:42.387470 29763 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:42.388546 29763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:42.389532 29763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:42.397925 29771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:42.398020 29763 server_base.cc:1034] running on GCE node
W20260811 11:59:42.397959 29770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:42.397927 29774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:42.398861 29763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:42.400089 29763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:42.401300 29763 hybrid_clock.cc:648] HybridClock initialized: now 1786449582401261 us; error 47 us; skew 500 ppm
Aug 11 11:59:42 dist-test-slave-zkpd krb5kdc[29750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449582, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:42.404248 29763 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 11:59:42.405243 29763 webserver.cc:460] Webserver started at http://127.0.0.1:37117/ using document root <none> and password file <none>
I20260811 11:59:42.405839 29763 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:42.405936 29763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:42.406183 29763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:42.408411 29763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "4f42283a9eec4d648bd3a2f4363de2c7"
format_stamp: "Formatted at 2026-08-11 11:59:42 on dist-test-slave-zkpd"
server_key: "0020def7e695b25ee12ab32f9d64b771"
server_key_iv: "5284de61d256d5c8bcf91140d29a7d94"
server_key_version: "encryptionkey@0"
I20260811 11:59:42.409020 29763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "4f42283a9eec4d648bd3a2f4363de2c7"
format_stamp: "Formatted at 2026-08-11 11:59:42 on dist-test-slave-zkpd"
server_key: "0020def7e695b25ee12ab32f9d64b771"
server_key_iv: "5284de61d256d5c8bcf91140d29a7d94"
server_key_version: "encryptionkey@0"
I20260811 11:59:42.412919 29763 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 11:59:42.415541 29782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:42.416853 29763 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 11:59:42.417009 29763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "4f42283a9eec4d648bd3a2f4363de2c7"
format_stamp: "Formatted at 2026-08-11 11:59:42 on dist-test-slave-zkpd"
server_key: "0020def7e695b25ee12ab32f9d64b771"
server_key_iv: "5284de61d256d5c8bcf91140d29a7d94"
server_key_version: "encryptionkey@0"
I20260811 11:59:42.417129 29763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:42.435642 29763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:42.438515 29763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:42.438719 29763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:42.446115 29763 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43535
I20260811 11:59:42.446146 29837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43535 every 8 connection(s)
I20260811 11:59:42.447110 29763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:59:42.449079 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 29763
I20260811 11:59:42.449247 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:59:42.449586 6718 external_mini_cluster.cc:1428] Setting key 2a0af4ddccbf9874cb009905b74e9d5b
I20260811 11:59:42.450379 29838 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:42.456218 29838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f42283a9eec4d648bd3a2f4363de2c7: Bootstrap starting.
I20260811 11:59:42.458515 29842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:42.451060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:39395 (local address 127.0.0.1:43535)
0811 11:59:42.451506 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:42.451522 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:42.451557 (+ 35us) server_negotiation.cc:407] Connection header received
0811 11:59:42.452262 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:42.452282 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:42.452616 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:42.452952 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:42.453351 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:42.454336 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:42.455231 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:42.455696 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:42.455880 (+ 184us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:42.456827 (+ 947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:42.456831 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:42.456844 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:42.457211 (+ 367us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:42.457263 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:42.457287 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:42.457427 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:42.457612 (+ 185us) server_negotiation.cc:299] Negotiation successful
0811 11:59:42.457746 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":152,"threads_started":1}
I20260811 11:59:42.458621 29838 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f42283a9eec4d648bd3a2f4363de2c7: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:42.459518 29838 log.cc:824] T 00000000000000000000000000000000 P 4f42283a9eec4d648bd3a2f4363de2c7: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:42.461704 29838 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f42283a9eec4d648bd3a2f4363de2c7: No bootstrap required, opened a new log
I20260811 11:59:42.461724 6718 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:59:42.465435 29838 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f42283a9eec4d648bd3a2f4363de2c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f42283a9eec4d648bd3a2f4363de2c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43535 } }
I20260811 11:59:42.465682 29838 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f42283a9eec4d648bd3a2f4363de2c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:42.465772 29838 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f42283a9eec4d648bd3a2f4363de2c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f42283a9eec4d648bd3a2f4363de2c7, State: Initialized, Role: FOLLOWER
I20260811 11:59:42.466275 29838 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f42283a9eec4d648bd3a2f4363de2c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f42283a9eec4d648bd3a2f4363de2c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43535 } }
I20260811 11:59:42.466447 29838 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f42283a9eec4d648bd3a2f4363de2c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:59:42.466522 29838 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f42283a9eec4d648bd3a2f4363de2c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:59:42.466631 29838 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f42283a9eec4d648bd3a2f4363de2c7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:42.467749 29842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:42.463719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:48581 (local address 127.0.0.1:43535)
0811 11:59:42.463845 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:42.463851 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:42.464059 (+ 208us) server_negotiation.cc:407] Connection header received
0811 11:59:42.464194 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:42.464197 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:42.464244 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:42.464327 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:42.464589 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:42.465084 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:42.465967 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:42.466297 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:42.466419 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:42.467308 (+ 889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:42.467311 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:42.467314 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:42.467325 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:42.467355 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:42.467367 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:42.467466 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:42.467578 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 11:59:42.467612 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 11:59:42.467864 29838 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f42283a9eec4d648bd3a2f4363de2c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f42283a9eec4d648bd3a2f4363de2c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43535 } }
I20260811 11:59:42.468264 29838 leader_election.cc:302] T 00000000000000000000000000000000 P 4f42283a9eec4d648bd3a2f4363de2c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f42283a9eec4d648bd3a2f4363de2c7; no voters:
I20260811 11:59:42.468598 29838 leader_election.cc:288] T 00000000000000000000000000000000 P 4f42283a9eec4d648bd3a2f4363de2c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:59:42.468705 29850 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f42283a9eec4d648bd3a2f4363de2c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:42.468961 29850 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f42283a9eec4d648bd3a2f4363de2c7 [term 1 LEADER]: Becoming Leader. State: Replica: 4f42283a9eec4d648bd3a2f4363de2c7, State: Running, Role: LEADER
I20260811 11:59:42.469247 29850 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f42283a9eec4d648bd3a2f4363de2c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f42283a9eec4d648bd3a2f4363de2c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43535 } }
I20260811 11:59:42.469800 29838 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f42283a9eec4d648bd3a2f4363de2c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:59:42.470974 29852 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f42283a9eec4d648bd3a2f4363de2c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f42283a9eec4d648bd3a2f4363de2c7. Latest consensus state: current_term: 1 leader_uuid: "4f42283a9eec4d648bd3a2f4363de2c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f42283a9eec4d648bd3a2f4363de2c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43535 } } }
I20260811 11:59:42.471011 29851 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f42283a9eec4d648bd3a2f4363de2c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f42283a9eec4d648bd3a2f4363de2c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f42283a9eec4d648bd3a2f4363de2c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43535 } } }
I20260811 11:59:42.471119 29851 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f42283a9eec4d648bd3a2f4363de2c7 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:42.471118 29852 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f42283a9eec4d648bd3a2f4363de2c7 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:42.471694 29859 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:59:42.475232 29859 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:59:42.480746 29859 catalog_manager.cc:1347] Generated new cluster ID: 2bb8bafc3d444d2eb1621f85527030eb
I20260811 11:59:42.480808 29859 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:59:42.489943 29859 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:59:42.491077 29859 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:59:42.499245 29859 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f42283a9eec4d648bd3a2f4363de2c7: Generated new TSK 0
I20260811 11:59:42.500025 29859 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:59:42.503129 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 29763
W20260811 11:59:42.512475 29748 connection.cc:535] client connection to 127.0.0.1:43535 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43535): Connection reset by peer (error 104)
2026-08-11T11:59:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1446 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:59:42 dist-test-slave-zkpd krb5kdc[29872](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:59:42 dist-test-slave-zkpd krb5kdc[29872](info): set up 2 sockets
Aug 11 11:59:42 dist-test-slave-zkpd krb5kdc[29872](info): commencing operation
krb5kdc: starting...
W20260811 11:59:43.584857 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:59:43 dist-test-slave-zkpd krb5kdc[29872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449583, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:59:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:59:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 11:59:43.710239 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41071
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41071
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:43.820905 29886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:43.821210 29886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:43.821292 29886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:43.825251 29886 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:59:43.825348 29886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:43.825388 29886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:43.825426 29886 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:59:43.825455 29886 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:59:43.830628 29886 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41071
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:43.831733 29886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:43.832777 29886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:43.841167 29892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:43.841161 29894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:43.841246 29886 server_base.cc:1034] running on GCE node
W20260811 11:59:43.841164 29891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:43.841956 29886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:43.842976 29886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:43.844164 29886 hybrid_clock.cc:648] HybridClock initialized: now 1786449583844144 us; error 30 us; skew 500 ppm
Aug 11 11:59:43 dist-test-slave-zkpd krb5kdc[29872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449583, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:43.847152 29886 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 11:59:43.848080 29886 webserver.cc:460] Webserver started at http://127.0.0.1:37949/ using document root <none> and password file <none>
I20260811 11:59:43.848657 29886 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:43.848770 29886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:43.849009 29886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:43.851076 29886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "b49de1be29d1490a934fb6c8265d80f0"
format_stamp: "Formatted at 2026-08-11 11:59:43 on dist-test-slave-zkpd"
server_key: "1f34b5354b529b9a9abd87375d5c97b2"
server_key_iv: "768c1af61d88f553add1524e0de95325"
server_key_version: "encryptionkey@0"
I20260811 11:59:43.851649 29886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "b49de1be29d1490a934fb6c8265d80f0"
format_stamp: "Formatted at 2026-08-11 11:59:43 on dist-test-slave-zkpd"
server_key: "1f34b5354b529b9a9abd87375d5c97b2"
server_key_iv: "768c1af61d88f553add1524e0de95325"
server_key_version: "encryptionkey@0"
I20260811 11:59:43.855304 29886 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 11:59:43.857777 29901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:43.859006 29886 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:59:43.859196 29886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "b49de1be29d1490a934fb6c8265d80f0"
format_stamp: "Formatted at 2026-08-11 11:59:43 on dist-test-slave-zkpd"
server_key: "1f34b5354b529b9a9abd87375d5c97b2"
server_key_iv: "768c1af61d88f553add1524e0de95325"
server_key_version: "encryptionkey@0"
I20260811 11:59:43.859308 29886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:43.863857 29886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:43.866808 29886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:43.866994 29886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:43.873638 29886 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41071
I20260811 11:59:43.873759 29961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41071 every 8 connection(s)
I20260811 11:59:43.874605 29886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:59:43.876159 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 29886
I20260811 11:59:43.876322 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:59:43.876639 6718 external_mini_cluster.cc:1428] Setting key 351e9f1f6178b1b0b097ad1d7776bd98
I20260811 11:59:43.877825 29962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:43.882153 29965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:43.877853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:44053 (local address 127.0.0.1:41071)
0811 11:59:43.878354 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:43.878371 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:43.878413 (+ 42us) server_negotiation.cc:407] Connection header received
0811 11:59:43.879294 (+ 881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:43.879317 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:43.879660 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:43.880062 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:43.880792 (+ 730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:43.880798 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:43.880813 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:43.880825 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:43.880889 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:43.880919 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:43.881062 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:43.881127 (+ 65us) server_negotiation.cc:299] Negotiation successful
0811 11:59:43.881288 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":119,"threads_started":1}
I20260811 11:59:43.884950 6718 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:59:43.885072 29962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b49de1be29d1490a934fb6c8265d80f0: Bootstrap starting.
I20260811 11:59:43.887463 29962 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b49de1be29d1490a934fb6c8265d80f0: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:43.888366 29962 log.cc:824] T 00000000000000000000000000000000 P b49de1be29d1490a934fb6c8265d80f0: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:43.888943 29965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:43.887226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:50023 (local address 127.0.0.1:41071)
0811 11:59:43.887338 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:43.887343 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:43.887528 (+ 185us) server_negotiation.cc:407] Connection header received
0811 11:59:43.887686 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:43.887689 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:43.887745 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:43.887830 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:43.888544 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:43.888547 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:43.888549 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:43.888561 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:43.888587 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:43.888604 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:43.888690 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:43.888793 (+ 103us) server_negotiation.cc:299] Negotiation successful
0811 11:59:43.888833 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 11:59:43.890440 29962 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b49de1be29d1490a934fb6c8265d80f0: No bootstrap required, opened a new log
I20260811 11:59:43.894014 29962 raft_consensus.cc:348] T 00000000000000000000000000000000 P b49de1be29d1490a934fb6c8265d80f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b49de1be29d1490a934fb6c8265d80f0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41071 } }
I20260811 11:59:43.894248 29962 raft_consensus.cc:374] T 00000000000000000000000000000000 P b49de1be29d1490a934fb6c8265d80f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:43.894323 29962 raft_consensus.cc:729] T 00000000000000000000000000000000 P b49de1be29d1490a934fb6c8265d80f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b49de1be29d1490a934fb6c8265d80f0, State: Initialized, Role: FOLLOWER
I20260811 11:59:43.894730 29962 consensus_queue.cc:260] T 00000000000000000000000000000000 P b49de1be29d1490a934fb6c8265d80f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b49de1be29d1490a934fb6c8265d80f0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41071 } }
I20260811 11:59:43.894892 29962 raft_consensus.cc:388] T 00000000000000000000000000000000 P b49de1be29d1490a934fb6c8265d80f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:59:43.894969 29962 raft_consensus.cc:482] T 00000000000000000000000000000000 P b49de1be29d1490a934fb6c8265d80f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:59:43.895083 29962 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b49de1be29d1490a934fb6c8265d80f0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:43.896077 29962 raft_consensus.cc:504] T 00000000000000000000000000000000 P b49de1be29d1490a934fb6c8265d80f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b49de1be29d1490a934fb6c8265d80f0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41071 } }
I20260811 11:59:43.896416 29962 leader_election.cc:302] T 00000000000000000000000000000000 P b49de1be29d1490a934fb6c8265d80f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b49de1be29d1490a934fb6c8265d80f0; no voters:
I20260811 11:59:43.896766 29962 leader_election.cc:288] T 00000000000000000000000000000000 P b49de1be29d1490a934fb6c8265d80f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:59:43.896934 29972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b49de1be29d1490a934fb6c8265d80f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:43.897164 29972 raft_consensus.cc:686] T 00000000000000000000000000000000 P b49de1be29d1490a934fb6c8265d80f0 [term 1 LEADER]: Becoming Leader. State: Replica: b49de1be29d1490a934fb6c8265d80f0, State: Running, Role: LEADER
I20260811 11:59:43.897495 29972 consensus_queue.cc:237] T 00000000000000000000000000000000 P b49de1be29d1490a934fb6c8265d80f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b49de1be29d1490a934fb6c8265d80f0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41071 } }
I20260811 11:59:43.897944 29962 sys_catalog.cc:564] T 00000000000000000000000000000000 P b49de1be29d1490a934fb6c8265d80f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:59:43.899251 29974 sys_catalog.cc:455] T 00000000000000000000000000000000 P b49de1be29d1490a934fb6c8265d80f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b49de1be29d1490a934fb6c8265d80f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b49de1be29d1490a934fb6c8265d80f0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41071 } } }
I20260811 11:59:43.899282 29975 sys_catalog.cc:455] T 00000000000000000000000000000000 P b49de1be29d1490a934fb6c8265d80f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b49de1be29d1490a934fb6c8265d80f0. Latest consensus state: current_term: 1 leader_uuid: "b49de1be29d1490a934fb6c8265d80f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b49de1be29d1490a934fb6c8265d80f0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41071 } } }
I20260811 11:59:43.899372 29974 sys_catalog.cc:458] T 00000000000000000000000000000000 P b49de1be29d1490a934fb6c8265d80f0 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:43.899394 29975 sys_catalog.cc:458] T 00000000000000000000000000000000 P b49de1be29d1490a934fb6c8265d80f0 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:43.900136 29983 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:59:43.903451 29983 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:59:43.909157 29983 catalog_manager.cc:1347] Generated new cluster ID: c3a4945891a2423a8b315af62bedb021
I20260811 11:59:43.909250 29983 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:59:43.918180 29983 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:59:43.918489 29983 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:59:43.925055 29983 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b49de1be29d1490a934fb6c8265d80f0: Generated new TSK 0
I20260811 11:59:43.925673 29983 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:59:43.931871 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 29886
2026-08-11T11:59:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1428 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:59:43 dist-test-slave-zkpd krb5kdc[29994](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:59:43 dist-test-slave-zkpd krb5kdc[29994](info): set up 2 sockets
Aug 11 11:59:43 dist-test-slave-zkpd krb5kdc[29994](info): commencing operation
krb5kdc: starting...
W20260811 11:59:45.012792 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.058s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:59:45 dist-test-slave-zkpd krb5kdc[29994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:59:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:59:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 11:59:45.146253 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34443
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:45.258358 30008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:45.258634 30008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:45.258708 30008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:45.263295 30008 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:59:45.263401 30008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:45.263448 30008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:45.263491 30008 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:59:45.263527 30008 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:59:45.269248 30008 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34443
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:45.270295 30008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:45.271376 30008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:45.279965 30014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:45.280030 30013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:45.280131 30008 server_base.cc:1034] running on GCE node
W20260811 11:59:45.279965 30016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:45.280812 30008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:45.281824 30008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:45.283020 30008 hybrid_clock.cc:648] HybridClock initialized: now 1786449585282983 us; error 49 us; skew 500 ppm
Aug 11 11:59:45 dist-test-slave-zkpd krb5kdc[29994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449585, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:45.285754 30008 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 11:59:45.286580 30008 webserver.cc:460] Webserver started at http://127.0.0.1:41279/ using document root <none> and password file <none>
I20260811 11:59:45.287133 30008 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:45.287223 30008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:45.287456 30008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:45.289505 30008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "a4fd686c06e04093947d315da69638ae"
format_stamp: "Formatted at 2026-08-11 11:59:45 on dist-test-slave-zkpd"
server_key: "e6079558e9247410fcb3d61a8007c2b2"
server_key_iv: "147718de1324f3a0662d45e0add48e9f"
server_key_version: "encryptionkey@0"
I20260811 11:59:45.290076 30008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "a4fd686c06e04093947d315da69638ae"
format_stamp: "Formatted at 2026-08-11 11:59:45 on dist-test-slave-zkpd"
server_key: "e6079558e9247410fcb3d61a8007c2b2"
server_key_iv: "147718de1324f3a0662d45e0add48e9f"
server_key_version: "encryptionkey@0"
I20260811 11:59:45.293856 30008 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 11:59:45.296304 30023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:45.297459 30008 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:59:45.297600 30008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "a4fd686c06e04093947d315da69638ae"
format_stamp: "Formatted at 2026-08-11 11:59:45 on dist-test-slave-zkpd"
server_key: "e6079558e9247410fcb3d61a8007c2b2"
server_key_iv: "147718de1324f3a0662d45e0add48e9f"
server_key_version: "encryptionkey@0"
I20260811 11:59:45.297709 30008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:45.312381 30008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:45.315253 30008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:45.315397 30008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:45.322428 30008 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34443
I20260811 11:59:45.322513 30082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34443 every 8 connection(s)
I20260811 11:59:45.323395 30008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:59:45.326473 30083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:45.332554 30083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4fd686c06e04093947d315da69638ae: Bootstrap starting.
I20260811 11:59:45.332934 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 30008
I20260811 11:59:45.333094 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:59:45.333377 6718 external_mini_cluster.cc:1428] Setting key cc2dbf72c30e5e3ad699fc30aa2de898
I20260811 11:59:45.335493 30083 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4fd686c06e04093947d315da69638ae: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:45.336491 30083 log.cc:824] T 00000000000000000000000000000000 P a4fd686c06e04093947d315da69638ae: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:45.338821 30083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4fd686c06e04093947d315da69638ae: No bootstrap required, opened a new log
Aug 11 11:59:45 dist-test-slave-zkpd krb5kdc[29994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449585, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 11:59:45.343602 30083 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4fd686c06e04093947d315da69638ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4fd686c06e04093947d315da69638ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34443 } }
I20260811 11:59:45.343930 30083 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4fd686c06e04093947d315da69638ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:45.344027 30083 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4fd686c06e04093947d315da69638ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4fd686c06e04093947d315da69638ae, State: Initialized, Role: FOLLOWER
I20260811 11:59:45.344566 30083 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4fd686c06e04093947d315da69638ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4fd686c06e04093947d315da69638ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34443 } }
I20260811 11:59:45.344735 30083 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4fd686c06e04093947d315da69638ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:59:45.344835 30083 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4fd686c06e04093947d315da69638ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:59:45.344976 30083 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4fd686c06e04093947d315da69638ae [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:45.345912 30083 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4fd686c06e04093947d315da69638ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4fd686c06e04093947d315da69638ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34443 } }
I20260811 11:59:45.346288 30083 leader_election.cc:302] T 00000000000000000000000000000000 P a4fd686c06e04093947d315da69638ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4fd686c06e04093947d315da69638ae; no voters:
I20260811 11:59:45.346669 30083 leader_election.cc:288] T 00000000000000000000000000000000 P a4fd686c06e04093947d315da69638ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:59:45.346707 30089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4fd686c06e04093947d315da69638ae [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:45.346915 30089 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4fd686c06e04093947d315da69638ae [term 1 LEADER]: Becoming Leader. State: Replica: a4fd686c06e04093947d315da69638ae, State: Running, Role: LEADER
I20260811 11:59:45.347304 30089 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4fd686c06e04093947d315da69638ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4fd686c06e04093947d315da69638ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34443 } }
I20260811 11:59:45.348083 30083 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4fd686c06e04093947d315da69638ae [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:59:45.349543 30090 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4fd686c06e04093947d315da69638ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4fd686c06e04093947d315da69638ae. Latest consensus state: current_term: 1 leader_uuid: "a4fd686c06e04093947d315da69638ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4fd686c06e04093947d315da69638ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34443 } } }
I20260811 11:59:45.349740 30090 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4fd686c06e04093947d315da69638ae [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:45.350278 30087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:45.334791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:41071 (local address 127.0.0.1:34443)
0811 11:59:45.335246 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:45.335269 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:45.335316 (+ 47us) server_negotiation.cc:407] Connection header received
0811 11:59:45.336291 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:45.336318 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:45.336788 (+ 470us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:45.337143 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:45.337910 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:45.339518 (+ 1608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:45.340413 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:45.341203 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:45.341383 (+ 180us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:45.344600 (+ 3217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:45.344605 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:45.344617 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:45.344643 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:45.346545 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:45.347155 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:45.347161 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:45.347167 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:45.347286 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:45.347685 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:45.347689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:45.347691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:45.348210 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:45.348468 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:45.348949 (+ 481us) server_negotiation.cc:299] Negotiation successful
0811 11:59:45.349187 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":125,"threads_started":1}
I20260811 11:59:45.350797 30091 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4fd686c06e04093947d315da69638ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4fd686c06e04093947d315da69638ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4fd686c06e04093947d315da69638ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34443 } } }
I20260811 11:59:45.350976 30091 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4fd686c06e04093947d315da69638ae [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:45.351287 30099 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:59:45.353528 6718 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:59:45.354799 30099 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:59:45.360757 30099 catalog_manager.cc:1347] Generated new cluster ID: ed2b0764c1e745baad2547524539d27c
I20260811 11:59:45.360864 30099 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:59:45.364099 30087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:45.355512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:40879 (local address 127.0.0.1:34443)
0811 11:59:45.355621 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:45.355624 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:45.355927 (+ 303us) server_negotiation.cc:407] Connection header received
0811 11:59:45.356037 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:45.356040 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:45.356080 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:45.356171 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:45.356799 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:45.357312 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:45.358103 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:45.358348 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:45.358463 (+ 115us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:45.360052 (+ 1589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:45.360056 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:45.360059 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:45.360084 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:45.361785 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:45.362382 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:45.362388 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:45.362391 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:45.362478 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:45.362916 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:45.362929 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:45.362933 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:45.363227 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:45.363350 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:45.363813 (+ 463us) server_negotiation.cc:299] Negotiation successful
0811 11:59:45.363943 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 11:59:45.370522 30099 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:59:45.371652 30099 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:59:45.387632 30099 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4fd686c06e04093947d315da69638ae: Generated new TSK 0
I20260811 11:59:45.388362 30099 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:59:45.395088 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 30008
W20260811 11:59:45.403328 29992 connection.cc:535] client connection to 127.0.0.1:34443 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34443): Connection reset by peer (error 104)
2026-08-11T11:59:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1462 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:59:45 dist-test-slave-zkpd krb5kdc[30116](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:59:45 dist-test-slave-zkpd krb5kdc[30116](info): set up 2 sockets
Aug 11 11:59:45 dist-test-slave-zkpd krb5kdc[30116](info): commencing operation
krb5kdc: starting...
W20260811 11:59:46.480880 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:59:46 dist-test-slave-zkpd krb5kdc[30116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449586, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:59:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:59:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 11:59:46.608448 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44563
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44563
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:46.723066 30130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:46.723346 30130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:46.723419 30130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:46.727371 30130 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:59:46.727458 30130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:46.727495 30130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:46.727552 30130 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:59:46.727592 30130 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:59:46.732661 30130 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44563
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44563
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30130
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:46.733637 30130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:46.734661 30130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:46.743016 30138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:46.743065 30135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:46.743023 30136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:46.743288 30130 server_base.cc:1034] running on GCE node
I20260811 11:59:46.743978 30130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:46.745008 30130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:46.746215 30130 hybrid_clock.cc:648] HybridClock initialized: now 1786449586746197 us; error 32 us; skew 500 ppm
Aug 11 11:59:46 dist-test-slave-zkpd krb5kdc[30116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449586, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:46.753314 30130 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 11:59:46.754146 30130 webserver.cc:460] Webserver started at http://127.0.0.1:37647/ using document root <none> and password file <none>
I20260811 11:59:46.754698 30130 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:46.754788 30130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:46.755079 30130 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:46.757110 30130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "5a688d44b92d4e518fd526bed9db174b"
format_stamp: "Formatted at 2026-08-11 11:59:46 on dist-test-slave-zkpd"
server_key: "361677f2095a73cf6d6cc78763a3110c"
server_key_iv: "f464e6bab25bba9805fdfa8842840fd4"
server_key_version: "encryptionkey@0"
I20260811 11:59:46.757704 30130 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "5a688d44b92d4e518fd526bed9db174b"
format_stamp: "Formatted at 2026-08-11 11:59:46 on dist-test-slave-zkpd"
server_key: "361677f2095a73cf6d6cc78763a3110c"
server_key_iv: "f464e6bab25bba9805fdfa8842840fd4"
server_key_version: "encryptionkey@0"
I20260811 11:59:46.761329 30130 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 11:59:46.763618 30145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:46.764832 30130 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:59:46.764982 30130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "5a688d44b92d4e518fd526bed9db174b"
format_stamp: "Formatted at 2026-08-11 11:59:46 on dist-test-slave-zkpd"
server_key: "361677f2095a73cf6d6cc78763a3110c"
server_key_iv: "f464e6bab25bba9805fdfa8842840fd4"
server_key_version: "encryptionkey@0"
I20260811 11:59:46.765106 30130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:46.774556 30130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:46.777253 30130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:46.777395 30130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:46.784226 30130 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44563
I20260811 11:59:46.784327 30202 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44563 every 8 connection(s)
I20260811 11:59:46.785158 30130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:59:46.785604 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 30130
I20260811 11:59:46.785760 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:59:46.786048 6718 external_mini_cluster.cc:1428] Setting key 1c3c5dd8237059e54746edad49893b26
I20260811 11:59:46.788591 30203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:46.795110 30203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a688d44b92d4e518fd526bed9db174b: Bootstrap starting.
I20260811 11:59:46.795445 30208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:46.787647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:37303 (local address 127.0.0.1:44563)
0811 11:59:46.788141 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:46.788156 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:46.788191 (+ 35us) server_negotiation.cc:407] Connection header received
0811 11:59:46.788881 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:46.788900 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:46.789226 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:46.789517 (+ 291us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:46.789924 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:46.790915 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:46.792233 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:46.792689 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:46.792802 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:46.793706 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:46.793712 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:46.793724 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:46.794078 (+ 354us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:46.794131 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:46.794153 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:46.794252 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:46.794485 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 11:59:46.794673 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":179,"threads_started":1}
I20260811 11:59:46.797513 30203 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5a688d44b92d4e518fd526bed9db174b: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:46.798368 30203 log.cc:824] T 00000000000000000000000000000000 P 5a688d44b92d4e518fd526bed9db174b: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:46.798461 6718 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:59:46.800442 30203 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5a688d44b92d4e518fd526bed9db174b: No bootstrap required, opened a new log
I20260811 11:59:46.804121 30203 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5a688d44b92d4e518fd526bed9db174b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a688d44b92d4e518fd526bed9db174b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44563 } }
I20260811 11:59:46.804414 30208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:46.800497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:48449 (local address 127.0.0.1:44563)
0811 11:59:46.800605 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:46.800609 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:46.800789 (+ 180us) server_negotiation.cc:407] Connection header received
0811 11:59:46.800950 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:46.800953 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:46.801000 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:46.801083 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:46.801340 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:46.801841 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:46.802651 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:46.802885 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:46.802993 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:46.803842 (+ 849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:46.803846 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:46.803848 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:46.803868 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:46.803935 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:46.803950 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:46.804058 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:46.804211 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 11:59:46.804248 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 11:59:46.804416 30203 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5a688d44b92d4e518fd526bed9db174b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:46.804576 30203 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5a688d44b92d4e518fd526bed9db174b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a688d44b92d4e518fd526bed9db174b, State: Initialized, Role: FOLLOWER
I20260811 11:59:46.804999 30203 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5a688d44b92d4e518fd526bed9db174b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a688d44b92d4e518fd526bed9db174b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44563 } }
I20260811 11:59:46.805167 30203 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5a688d44b92d4e518fd526bed9db174b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:59:46.805254 30203 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5a688d44b92d4e518fd526bed9db174b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:59:46.805377 30203 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5a688d44b92d4e518fd526bed9db174b [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:46.806269 30203 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5a688d44b92d4e518fd526bed9db174b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a688d44b92d4e518fd526bed9db174b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44563 } }
I20260811 11:59:46.806617 30203 leader_election.cc:302] T 00000000000000000000000000000000 P 5a688d44b92d4e518fd526bed9db174b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5a688d44b92d4e518fd526bed9db174b; no voters:
I20260811 11:59:46.806929 30203 leader_election.cc:288] T 00000000000000000000000000000000 P 5a688d44b92d4e518fd526bed9db174b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:59:46.807050 30216 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5a688d44b92d4e518fd526bed9db174b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:46.807305 30216 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5a688d44b92d4e518fd526bed9db174b [term 1 LEADER]: Becoming Leader. State: Replica: 5a688d44b92d4e518fd526bed9db174b, State: Running, Role: LEADER
I20260811 11:59:46.807605 30216 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5a688d44b92d4e518fd526bed9db174b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a688d44b92d4e518fd526bed9db174b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44563 } }
I20260811 11:59:46.808287 30203 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5a688d44b92d4e518fd526bed9db174b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:59:46.809553 30217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a688d44b92d4e518fd526bed9db174b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5a688d44b92d4e518fd526bed9db174b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a688d44b92d4e518fd526bed9db174b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44563 } } }
I20260811 11:59:46.809715 30217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a688d44b92d4e518fd526bed9db174b [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:46.810129 30226 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:59:46.810079 30218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5a688d44b92d4e518fd526bed9db174b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5a688d44b92d4e518fd526bed9db174b. Latest consensus state: current_term: 1 leader_uuid: "5a688d44b92d4e518fd526bed9db174b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5a688d44b92d4e518fd526bed9db174b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44563 } } }
I20260811 11:59:46.810245 30218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5a688d44b92d4e518fd526bed9db174b [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:46.813100 30226 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:59:46.818320 30226 catalog_manager.cc:1347] Generated new cluster ID: ecd3033fd65b49cbbc881b029ca3281a
I20260811 11:59:46.818383 30226 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:59:46.839229 30226 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:59:46.840318 30226 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:59:46.850821 30226 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5a688d44b92d4e518fd526bed9db174b: Generated new TSK 0
I20260811 11:59:46.851600 30226 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:59:46.859073 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 30130
W20260811 11:59:46.867533 30114 connection.cc:535] client connection to 127.0.0.1:44563 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44563): Connection reset by peer (error 104)
2026-08-11T11:59:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1464 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:59:46 dist-test-slave-zkpd krb5kdc[30240](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:59:46 dist-test-slave-zkpd krb5kdc[30240](info): set up 2 sockets
Aug 11 11:59:46 dist-test-slave-zkpd krb5kdc[30240](info): commencing operation
krb5kdc: starting...
W20260811 11:59:47.936673 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:59:48 dist-test-slave-zkpd krb5kdc[30240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449588, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:59:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:59:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 11:59:48.058607 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40309
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36485
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40309
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:48.171320 30254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:48.171633 30254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:48.171720 30254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:48.175668 30254 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:59:48.175760 30254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:48.175798 30254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:48.175835 30254 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:59:48.175864 30254 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:59:48.181005 30254 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36485
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40309
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449319379472-6718-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40309
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:48.182045 30254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:48.183091 30254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:48.191367 30262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:48.191365 30260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:48.191493 30254 server_base.cc:1034] running on GCE node
W20260811 11:59:48.191367 30259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:48.192291 30254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:48.193347 30254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:48.194511 30254 hybrid_clock.cc:648] HybridClock initialized: now 1786449588194491 us; error 30 us; skew 500 ppm
Aug 11 11:59:48 dist-test-slave-zkpd krb5kdc[30240](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449588, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:48.197242 30254 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 11:59:48.198125 30254 webserver.cc:460] Webserver started at http://127.0.0.1:35041/ using document root <none> and password file <none>
I20260811 11:59:48.198684 30254 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:48.198776 30254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:48.199018 30254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:48.201092 30254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "cfbf1e8989e049789a9861e2fc75fbcc"
format_stamp: "Formatted at 2026-08-11 11:59:48 on dist-test-slave-zkpd"
server_key: "75893a686498e4f747b23bea73394c48"
server_key_iv: "bb7577186550646804c978b0391a28a5"
server_key_version: "encryptionkey@0"
I20260811 11:59:48.201664 30254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "cfbf1e8989e049789a9861e2fc75fbcc"
format_stamp: "Formatted at 2026-08-11 11:59:48 on dist-test-slave-zkpd"
server_key: "75893a686498e4f747b23bea73394c48"
server_key_iv: "bb7577186550646804c978b0391a28a5"
server_key_version: "encryptionkey@0"
I20260811 11:59:48.205276 30254 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 11:59:48.207582 30270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:48.208746 30254 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 11:59:48.208895 30254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "cfbf1e8989e049789a9861e2fc75fbcc"
format_stamp: "Formatted at 2026-08-11 11:59:48 on dist-test-slave-zkpd"
server_key: "75893a686498e4f747b23bea73394c48"
server_key_iv: "bb7577186550646804c978b0391a28a5"
server_key_version: "encryptionkey@0"
I20260811 11:59:48.209022 30254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:48.213483 30254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:48.216231 30254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:48.216431 30254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:48.223549 30254 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40309
I20260811 11:59:48.223565 30323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40309 every 8 connection(s)
I20260811 11:59:48.224521 30254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:59:48.227221 30324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:48.232540 30324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfbf1e8989e049789a9861e2fc75fbcc: Bootstrap starting.
I20260811 11:59:48.234594 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 30254
I20260811 11:59:48.234753 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:59:48.234913 30324 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cfbf1e8989e049789a9861e2fc75fbcc: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:48.235078 6718 external_mini_cluster.cc:1428] Setting key 5fa310424eb2cedd6d9811c059136662
I20260811 11:59:48.235824 30324 log.cc:824] T 00000000000000000000000000000000 P cfbf1e8989e049789a9861e2fc75fbcc: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:48.237694 30324 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cfbf1e8989e049789a9861e2fc75fbcc: No bootstrap required, opened a new log
I20260811 11:59:48.239979 30328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:48.236186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:45889 (local address 127.0.0.1:40309)
0811 11:59:48.236649 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:48.236664 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:48.236699 (+ 35us) server_negotiation.cc:407] Connection header received
0811 11:59:48.237343 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:48.237360 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:48.237631 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:48.237962 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:48.238744 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:48.238749 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:48.238761 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:48.238770 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:48.238824 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:48.238844 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:48.238939 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:48.239063 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 11:59:48.239233 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":144,"threads_started":1}
I20260811 11:59:48.242028 30324 raft_consensus.cc:348] T 00000000000000000000000000000000 P cfbf1e8989e049789a9861e2fc75fbcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfbf1e8989e049789a9861e2fc75fbcc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40309 } }
I20260811 11:59:48.242291 30324 raft_consensus.cc:374] T 00000000000000000000000000000000 P cfbf1e8989e049789a9861e2fc75fbcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:48.242360 30324 raft_consensus.cc:729] T 00000000000000000000000000000000 P cfbf1e8989e049789a9861e2fc75fbcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cfbf1e8989e049789a9861e2fc75fbcc, State: Initialized, Role: FOLLOWER
I20260811 11:59:48.242772 6718 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:59:48.242859 30324 consensus_queue.cc:260] T 00000000000000000000000000000000 P cfbf1e8989e049789a9861e2fc75fbcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfbf1e8989e049789a9861e2fc75fbcc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40309 } }
I20260811 11:59:48.243027 30324 raft_consensus.cc:388] T 00000000000000000000000000000000 P cfbf1e8989e049789a9861e2fc75fbcc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:59:48.243093 30324 raft_consensus.cc:482] T 00000000000000000000000000000000 P cfbf1e8989e049789a9861e2fc75fbcc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:59:48.243184 30324 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cfbf1e8989e049789a9861e2fc75fbcc [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:48.244108 30324 raft_consensus.cc:504] T 00000000000000000000000000000000 P cfbf1e8989e049789a9861e2fc75fbcc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfbf1e8989e049789a9861e2fc75fbcc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40309 } }
I20260811 11:59:48.244447 30324 leader_election.cc:302] T 00000000000000000000000000000000 P cfbf1e8989e049789a9861e2fc75fbcc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cfbf1e8989e049789a9861e2fc75fbcc; no voters:
I20260811 11:59:48.244760 30324 leader_election.cc:288] T 00000000000000000000000000000000 P cfbf1e8989e049789a9861e2fc75fbcc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:59:48.245035 30333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cfbf1e8989e049789a9861e2fc75fbcc [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:48.245296 30333 raft_consensus.cc:686] T 00000000000000000000000000000000 P cfbf1e8989e049789a9861e2fc75fbcc [term 1 LEADER]: Becoming Leader. State: Replica: cfbf1e8989e049789a9861e2fc75fbcc, State: Running, Role: LEADER
I20260811 11:59:48.245656 30333 consensus_queue.cc:237] T 00000000000000000000000000000000 P cfbf1e8989e049789a9861e2fc75fbcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfbf1e8989e049789a9861e2fc75fbcc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40309 } }
I20260811 11:59:48.246025 30324 sys_catalog.cc:564] T 00000000000000000000000000000000 P cfbf1e8989e049789a9861e2fc75fbcc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:59:48.246866 30328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:48.244888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.39:42631 (local address 127.0.0.1:40309)
0811 11:59:48.244990 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:48.244994 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:48.245293 (+ 299us) server_negotiation.cc:407] Connection header received
0811 11:59:48.245520 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:48.245523 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:48.245556 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:48.245641 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:48.246456 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:48.246459 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:48.246462 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:48.246474 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:48.246499 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:48.246512 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:48.246595 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:48.246673 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 11:59:48.246711 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260811 11:59:48.246927 30333 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfbf1e8989e049789a9861e2fc75fbcc [sys.catalog]: SysCatalogTable state changed. Reason: New leader cfbf1e8989e049789a9861e2fc75fbcc. Latest consensus state: current_term: 1 leader_uuid: "cfbf1e8989e049789a9861e2fc75fbcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfbf1e8989e049789a9861e2fc75fbcc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40309 } } }
I20260811 11:59:48.247036 30333 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfbf1e8989e049789a9861e2fc75fbcc [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:48.247570 30335 sys_catalog.cc:455] T 00000000000000000000000000000000 P cfbf1e8989e049789a9861e2fc75fbcc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cfbf1e8989e049789a9861e2fc75fbcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cfbf1e8989e049789a9861e2fc75fbcc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40309 } } }
I20260811 11:59:48.247732 30335 sys_catalog.cc:458] T 00000000000000000000000000000000 P cfbf1e8989e049789a9861e2fc75fbcc [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:48.248082 30347 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:59:48.251153 30347 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:59:48.256251 30347 catalog_manager.cc:1347] Generated new cluster ID: 149f375ca3fb483d8f61cd2a1f4b73f9
I20260811 11:59:48.256314 30347 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:59:48.290334 30347 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:59:48.290696 30347 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:59:48.297612 30347 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cfbf1e8989e049789a9861e2fc75fbcc: Generated new TSK 0
I20260811 11:59:48.298347 30347 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:59:48.303745 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 30254
2026-08-11T11:59:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1443 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:59:48 dist-test-slave-zkpd krb5kdc[30357](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:59:48 dist-test-slave-zkpd krb5kdc[30357](info): set up 2 sockets
Aug 11 11:59:48 dist-test-slave-zkpd krb5kdc[30357](info): commencing operation
krb5kdc: starting...
W20260811 11:59:49.398510 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.073s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:59:49 dist-test-slave-zkpd krb5kdc[30357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:59:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:59:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
I20260811 11:59:49.525051 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:35363
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:38631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:35363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:49.635013 30373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:49.635246 30373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:49.635285 30373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:49.639384 30373 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:59:49.639439 30373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:49.639458 30373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:49.639473 30373 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:59:49.639518 30373 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:59:49.644671 30373 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:38631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:35363
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:35363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:49.645690 30373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:49.646705 30373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:49.655277 30379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:49.655294 30381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:49.655277 30378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:49.655397 30373 server_base.cc:1034] running on GCE node
I20260811 11:59:49.656157 30373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:49.657202 30373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:49.658394 30373 hybrid_clock.cc:648] HybridClock initialized: now 1786449589658356 us; error 47 us; skew 500 ppm
Aug 11 11:59:49 dist-test-slave-zkpd krb5kdc[30357](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449589, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:49.661217 30373 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:59:49.662124 30373 webserver.cc:460] Webserver started at http://127.6.143.190:43491/ using document root <none> and password file <none>
I20260811 11:59:49.662680 30373 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:49.662768 30373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:49.663007 30373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:49.665050 30373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "0af75bf5d0fc4a6fbf6fdb7394773e17"
format_stamp: "Formatted at 2026-08-11 11:59:49 on dist-test-slave-zkpd"
server_key: "977939c92165a1bc92a2dfa294b8b649"
server_key_iv: "db8d96fa0949d28360166f9313663d3a"
server_key_version: "encryptionkey@0"
I20260811 11:59:49.665602 30373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "0af75bf5d0fc4a6fbf6fdb7394773e17"
format_stamp: "Formatted at 2026-08-11 11:59:49 on dist-test-slave-zkpd"
server_key: "977939c92165a1bc92a2dfa294b8b649"
server_key_iv: "db8d96fa0949d28360166f9313663d3a"
server_key_version: "encryptionkey@0"
I20260811 11:59:49.669381 30373 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 11:59:49.671726 30389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:49.672958 30373 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:59:49.673096 30373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "0af75bf5d0fc4a6fbf6fdb7394773e17"
format_stamp: "Formatted at 2026-08-11 11:59:49 on dist-test-slave-zkpd"
server_key: "977939c92165a1bc92a2dfa294b8b649"
server_key_iv: "db8d96fa0949d28360166f9313663d3a"
server_key_version: "encryptionkey@0"
I20260811 11:59:49.673202 30373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:49.692723 30373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:49.695856 30373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:49.696074 30373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:49.703967 30373 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:35363
I20260811 11:59:49.704010 30448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:35363 every 8 connection(s)
I20260811 11:59:49.704908 30373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:59:49.707623 30449 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:49.711010 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 30373
I20260811 11:59:49.711176 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:59:49.711488 6718 external_mini_cluster.cc:1428] Setting key bd5313e30b4f8b96b888f588be929c63
I20260811 11:59:49.714310 30449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0af75bf5d0fc4a6fbf6fdb7394773e17: Bootstrap starting.
I20260811 11:59:49.716565 30449 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0af75bf5d0fc4a6fbf6fdb7394773e17: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:49.717326 30449 log.cc:824] T 00000000000000000000000000000000 P 0af75bf5d0fc4a6fbf6fdb7394773e17: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:49.719324 30449 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0af75bf5d0fc4a6fbf6fdb7394773e17: No bootstrap required, opened a new log
Aug 11 11:59:49 dist-test-slave-zkpd krb5kdc[30357](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449589, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:59:49.723332 30449 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0af75bf5d0fc4a6fbf6fdb7394773e17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af75bf5d0fc4a6fbf6fdb7394773e17" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 35363 } }
I20260811 11:59:49.723609 30449 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0af75bf5d0fc4a6fbf6fdb7394773e17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:49.723668 30449 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0af75bf5d0fc4a6fbf6fdb7394773e17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0af75bf5d0fc4a6fbf6fdb7394773e17, State: Initialized, Role: FOLLOWER
I20260811 11:59:49.724241 30449 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0af75bf5d0fc4a6fbf6fdb7394773e17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af75bf5d0fc4a6fbf6fdb7394773e17" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 35363 } }
I20260811 11:59:49.724424 30449 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0af75bf5d0fc4a6fbf6fdb7394773e17 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:59:49.724496 30449 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0af75bf5d0fc4a6fbf6fdb7394773e17 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:59:49.724613 30449 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0af75bf5d0fc4a6fbf6fdb7394773e17 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:49.725760 30449 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0af75bf5d0fc4a6fbf6fdb7394773e17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af75bf5d0fc4a6fbf6fdb7394773e17" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 35363 } }
I20260811 11:59:49.726125 30449 leader_election.cc:302] T 00000000000000000000000000000000 P 0af75bf5d0fc4a6fbf6fdb7394773e17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0af75bf5d0fc4a6fbf6fdb7394773e17; no voters:
I20260811 11:59:49.726455 30449 leader_election.cc:288] T 00000000000000000000000000000000 P 0af75bf5d0fc4a6fbf6fdb7394773e17 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:59:49.726599 30454 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0af75bf5d0fc4a6fbf6fdb7394773e17 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:49.726897 30454 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0af75bf5d0fc4a6fbf6fdb7394773e17 [term 1 LEADER]: Becoming Leader. State: Replica: 0af75bf5d0fc4a6fbf6fdb7394773e17, State: Running, Role: LEADER
I20260811 11:59:49.727144 30452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:49.712891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34009 (local address 127.6.143.190:35363)
0811 11:59:49.713315 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:49.713331 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:49.713365 (+ 34us) server_negotiation.cc:407] Connection header received
0811 11:59:49.713995 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:49.714013 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:49.714348 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:49.714724 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:49.715555 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:49.716666 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:49.717553 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:49.718163 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:49.718328 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:49.720920 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:49.720926 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:49.720945 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:49.720975 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:49.723550 (+ 2575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:49.724270 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:49.724276 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:49.724282 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:49.724452 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:49.724920 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:49.724927 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:49.724932 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:49.725439 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:49.725665 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:49.725934 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 11:59:49.726165 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":136,"threads_started":1}
I20260811 11:59:49.727429 30454 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0af75bf5d0fc4a6fbf6fdb7394773e17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af75bf5d0fc4a6fbf6fdb7394773e17" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 35363 } }
I20260811 11:59:49.727716 30449 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0af75bf5d0fc4a6fbf6fdb7394773e17 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:59:49.728869 30455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0af75bf5d0fc4a6fbf6fdb7394773e17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0af75bf5d0fc4a6fbf6fdb7394773e17. Latest consensus state: current_term: 1 leader_uuid: "0af75bf5d0fc4a6fbf6fdb7394773e17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af75bf5d0fc4a6fbf6fdb7394773e17" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 35363 } } }
I20260811 11:59:49.728988 30455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0af75bf5d0fc4a6fbf6fdb7394773e17 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:49.729439 30456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0af75bf5d0fc4a6fbf6fdb7394773e17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0af75bf5d0fc4a6fbf6fdb7394773e17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0af75bf5d0fc4a6fbf6fdb7394773e17" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 35363 } } }
I20260811 11:59:49.729549 30456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0af75bf5d0fc4a6fbf6fdb7394773e17 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:49.730829 6718 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:59:49.730844 30458 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:59:49.734153 30458 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:59:49.743647 30458 catalog_manager.cc:1347] Generated new cluster ID: a735cefca8954418a571ac87a8310df8
I20260811 11:59:49.744004 30458 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:59:49.746532 30452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:49.736375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35285 (local address 127.6.143.190:35363)
0811 11:59:49.736491 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:49.736496 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:49.738811 (+ 2315us) server_negotiation.cc:407] Connection header received
0811 11:59:49.738848 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:49.738850 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:49.738885 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:49.738983 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:49.739652 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:49.740226 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:49.741014 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:49.741239 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:49.741344 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:49.742770 (+ 1426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:49.742774 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:49.742776 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:49.742797 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:49.744624 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:49.745220 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:49.745224 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:49.745226 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:49.745306 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:49.745668 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:49.745672 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:49.745674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:49.745853 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:49.745990 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:49.746303 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 11:59:49.746388 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 11:59:49.754776 30458 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:59:49.756070 30458 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:59:49.763164 30458 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0af75bf5d0fc4a6fbf6fdb7394773e17: Generated new TSK 0
I20260811 11:59:49.763959 30458 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:59:49.767875 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 30373
W20260811 11:59:49.776837 30355 connection.cc:535] client connection to 127.6.143.190:35363 recv error: Network error: failed to read from TLS socket (remote: 127.6.143.190:35363): Connection reset by peer (error 104)
2026-08-11T11:59:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1465 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:59:49 dist-test-slave-zkpd krb5kdc[30480](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:59:49 dist-test-slave-zkpd krb5kdc[30480](info): set up 2 sockets
Aug 11 11:59:49 dist-test-slave-zkpd krb5kdc[30480](info): commencing operation
krb5kdc: starting...
W20260811 11:59:50.860949 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:59:50 dist-test-slave-zkpd krb5kdc[30480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449590, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:59:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:59:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
I20260811 11:59:50.986532 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:41577
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:44813
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:41577
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:51.096626 30494 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:51.096915 30494 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:51.097000 30494 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:51.100983 30494 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:59:51.101064 30494 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:51.101104 30494 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:51.101162 30494 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:59:51.101202 30494 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:59:51.106377 30494 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:44813
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:41577
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:41577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30494
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:51.107407 30494 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:51.108446 30494 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:51.116756 30500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:51.116775 30499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:51.116763 30502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:51.116981 30494 server_base.cc:1034] running on GCE node
I20260811 11:59:51.117617 30494 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:51.118618 30494 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:51.119832 30494 hybrid_clock.cc:648] HybridClock initialized: now 1786449591119813 us; error 31 us; skew 500 ppm
Aug 11 11:59:51 dist-test-slave-zkpd krb5kdc[30480](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449591, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:51.122740 30494 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:59:51.123673 30494 webserver.cc:460] Webserver started at http://127.6.143.190:36157/ using document root <none> and password file <none>
I20260811 11:59:51.124274 30494 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:51.124365 30494 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:51.124605 30494 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:51.126652 30494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "709cd1b801b947f68eabec4b5d60bb71"
format_stamp: "Formatted at 2026-08-11 11:59:51 on dist-test-slave-zkpd"
server_key: "2f9d71cd8e7c476f96014dc5c471e0a5"
server_key_iv: "de68ba7e5dd3783611dd41b32be46761"
server_key_version: "encryptionkey@0"
I20260811 11:59:51.127223 30494 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "709cd1b801b947f68eabec4b5d60bb71"
format_stamp: "Formatted at 2026-08-11 11:59:51 on dist-test-slave-zkpd"
server_key: "2f9d71cd8e7c476f96014dc5c471e0a5"
server_key_iv: "de68ba7e5dd3783611dd41b32be46761"
server_key_version: "encryptionkey@0"
I20260811 11:59:51.131083 30494 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 11:59:51.133627 30511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:51.134814 30494 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 11:59:51.134961 30494 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "709cd1b801b947f68eabec4b5d60bb71"
format_stamp: "Formatted at 2026-08-11 11:59:51 on dist-test-slave-zkpd"
server_key: "2f9d71cd8e7c476f96014dc5c471e0a5"
server_key_iv: "de68ba7e5dd3783611dd41b32be46761"
server_key_version: "encryptionkey@0"
I20260811 11:59:51.135072 30494 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:51.144618 30494 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:51.147326 30494 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:51.147545 30494 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:51.154956 30494 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:41577
I20260811 11:59:51.154983 30568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:41577 every 8 connection(s)
I20260811 11:59:51.155920 30494 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:59:51.158779 30569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:51.162729 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 30494
I20260811 11:59:51.162879 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:59:51.163174 6718 external_mini_cluster.cc:1428] Setting key 05b75be7a4566d45bc2b67efee5bca8f
I20260811 11:59:51.164906 30569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 709cd1b801b947f68eabec4b5d60bb71: Bootstrap starting.
I20260811 11:59:51.167285 30569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 709cd1b801b947f68eabec4b5d60bb71: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:51.168195 30569 log.cc:824] T 00000000000000000000000000000000 P 709cd1b801b947f68eabec4b5d60bb71: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:51.170382 30569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 709cd1b801b947f68eabec4b5d60bb71: No bootstrap required, opened a new log
I20260811 11:59:51.172349 30572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:51.164496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58389 (local address 127.6.143.190:41577)
0811 11:59:51.164876 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:51.164901 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:51.164938 (+ 37us) server_negotiation.cc:407] Connection header received
0811 11:59:51.165644 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:51.165661 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:51.165987 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:51.166256 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:51.166597 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:51.167639 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:51.168834 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:51.169341 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:51.169468 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:51.170362 (+ 894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:51.170367 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:51.170379 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:51.170805 (+ 426us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:51.170858 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:51.170881 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:51.170985 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:51.171193 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 11:59:51.171352 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":123,"threads_started":1}
I20260811 11:59:51.173916 30569 raft_consensus.cc:348] T 00000000000000000000000000000000 P 709cd1b801b947f68eabec4b5d60bb71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "709cd1b801b947f68eabec4b5d60bb71" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 41577 } }
I20260811 11:59:51.174165 30569 raft_consensus.cc:374] T 00000000000000000000000000000000 P 709cd1b801b947f68eabec4b5d60bb71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:51.174235 30569 raft_consensus.cc:729] T 00000000000000000000000000000000 P 709cd1b801b947f68eabec4b5d60bb71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 709cd1b801b947f68eabec4b5d60bb71, State: Initialized, Role: FOLLOWER
I20260811 11:59:51.174638 30569 consensus_queue.cc:260] T 00000000000000000000000000000000 P 709cd1b801b947f68eabec4b5d60bb71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "709cd1b801b947f68eabec4b5d60bb71" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 41577 } }
I20260811 11:59:51.174817 30569 raft_consensus.cc:388] T 00000000000000000000000000000000 P 709cd1b801b947f68eabec4b5d60bb71 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:59:51.174896 30569 raft_consensus.cc:482] T 00000000000000000000000000000000 P 709cd1b801b947f68eabec4b5d60bb71 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:59:51.175038 30569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 709cd1b801b947f68eabec4b5d60bb71 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:51.175452 6718 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:59:51.176083 30569 raft_consensus.cc:504] T 00000000000000000000000000000000 P 709cd1b801b947f68eabec4b5d60bb71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "709cd1b801b947f68eabec4b5d60bb71" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 41577 } }
I20260811 11:59:51.176417 30569 leader_election.cc:302] T 00000000000000000000000000000000 P 709cd1b801b947f68eabec4b5d60bb71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 709cd1b801b947f68eabec4b5d60bb71; no voters:
I20260811 11:59:51.176777 30569 leader_election.cc:288] T 00000000000000000000000000000000 P 709cd1b801b947f68eabec4b5d60bb71 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:59:51.176882 30576 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 709cd1b801b947f68eabec4b5d60bb71 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:51.177080 30576 raft_consensus.cc:686] T 00000000000000000000000000000000 P 709cd1b801b947f68eabec4b5d60bb71 [term 1 LEADER]: Becoming Leader. State: Replica: 709cd1b801b947f68eabec4b5d60bb71, State: Running, Role: LEADER
I20260811 11:59:51.177488 30576 consensus_queue.cc:237] T 00000000000000000000000000000000 P 709cd1b801b947f68eabec4b5d60bb71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "709cd1b801b947f68eabec4b5d60bb71" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 41577 } }
I20260811 11:59:51.178107 30569 sys_catalog.cc:564] T 00000000000000000000000000000000 P 709cd1b801b947f68eabec4b5d60bb71 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:59:51.178794 30581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 709cd1b801b947f68eabec4b5d60bb71 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 709cd1b801b947f68eabec4b5d60bb71. Latest consensus state: current_term: 1 leader_uuid: "709cd1b801b947f68eabec4b5d60bb71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "709cd1b801b947f68eabec4b5d60bb71" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 41577 } } }
I20260811 11:59:51.178921 30581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 709cd1b801b947f68eabec4b5d60bb71 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:51.179605 30580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 709cd1b801b947f68eabec4b5d60bb71 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "709cd1b801b947f68eabec4b5d60bb71" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "709cd1b801b947f68eabec4b5d60bb71" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 41577 } } }
I20260811 11:59:51.179756 30580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 709cd1b801b947f68eabec4b5d60bb71 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:51.180449 30586 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:59:51.182617 30572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:51.177672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54829 (local address 127.6.143.190:41577)
0811 11:59:51.177773 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:51.177777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:51.178176 (+ 399us) server_negotiation.cc:407] Connection header received
0811 11:59:51.178333 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:51.178335 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:51.178378 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:51.178468 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:51.178795 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:51.179660 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:51.180876 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:51.181116 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:51.181226 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:51.182095 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:51.182101 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:51.182105 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:51.182119 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:51.182147 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:51.182164 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:51.182249 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:51.182444 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 11:59:51.182481 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":31}
I20260811 11:59:51.183593 30586 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:59:51.189507 30586 catalog_manager.cc:1347] Generated new cluster ID: 39fa5fd3c4214852ad944d3593e8a65c
I20260811 11:59:51.189584 30586 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:59:51.200919 30586 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:59:51.202136 30586 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:59:51.210294 30586 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 709cd1b801b947f68eabec4b5d60bb71: Generated new TSK 0
I20260811 11:59:51.211090 30586 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:59:51.219465 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 30494
W20260811 11:59:51.227687 30478 connection.cc:535] client connection to 127.6.143.190:41577 recv error: Network error: failed to read from TLS socket (remote: 127.6.143.190:41577): Connection reset by peer (error 104)
2026-08-11T11:59:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1450 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:59:51 dist-test-slave-zkpd krb5kdc[30603](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:59:51 dist-test-slave-zkpd krb5kdc[30603](info): set up 2 sockets
Aug 11 11:59:51 dist-test-slave-zkpd krb5kdc[30603](info): commencing operation
krb5kdc: starting...
W20260811 11:59:52.302235 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:59:52 dist-test-slave-zkpd krb5kdc[30603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449592, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:59:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:59:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
I20260811 11:59:52.429037 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:43559
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:41235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:43559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:52.540508 30616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:52.540866 30616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:52.540954 30616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:52.544884 30616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:59:52.544986 30616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:52.545025 30616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:52.545084 30616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:59:52.545125 30616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:59:52.550174 30616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:41235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:43559
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:43559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:52.551183 30616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:52.552244 30616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:52.560509 30622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:52.560492 30624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:52.560521 30621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:52.560813 30616 server_base.cc:1034] running on GCE node
I20260811 11:59:52.561390 30616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:52.562413 30616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:52.563604 30616 hybrid_clock.cc:648] HybridClock initialized: now 1786449592563559 us; error 56 us; skew 500 ppm
Aug 11 11:59:52 dist-test-slave-zkpd krb5kdc[30603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449592, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:52.570551 30616 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:59:52.571417 30616 webserver.cc:460] Webserver started at http://127.6.143.190:40081/ using document root <none> and password file <none>
I20260811 11:59:52.572062 30616 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:52.572152 30616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:52.572373 30616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:52.574388 30616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "edadd11dc3a54ee9be36180cd1f15e60"
format_stamp: "Formatted at 2026-08-11 11:59:52 on dist-test-slave-zkpd"
server_key: "e91c9abda172f2d25c3328b6b309fcb2"
server_key_iv: "361911db08b7bbfc7fc92e87253ff993"
server_key_version: "encryptionkey@0"
I20260811 11:59:52.574966 30616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "edadd11dc3a54ee9be36180cd1f15e60"
format_stamp: "Formatted at 2026-08-11 11:59:52 on dist-test-slave-zkpd"
server_key: "e91c9abda172f2d25c3328b6b309fcb2"
server_key_iv: "361911db08b7bbfc7fc92e87253ff993"
server_key_version: "encryptionkey@0"
I20260811 11:59:52.578533 30616 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260811 11:59:52.580942 30632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:52.582191 30616 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:59:52.582343 30616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "edadd11dc3a54ee9be36180cd1f15e60"
format_stamp: "Formatted at 2026-08-11 11:59:52 on dist-test-slave-zkpd"
server_key: "e91c9abda172f2d25c3328b6b309fcb2"
server_key_iv: "361911db08b7bbfc7fc92e87253ff993"
server_key_version: "encryptionkey@0"
I20260811 11:59:52.582453 30616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:52.586948 30616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:52.589962 30616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:52.590173 30616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:52.597872 30616 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:43559
I20260811 11:59:52.597906 30689 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:43559 every 8 connection(s)
I20260811 11:59:52.598789 30616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:59:52.601738 30691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:52.605641 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 30616
I20260811 11:59:52.605804 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:59:52.606098 6718 external_mini_cluster.cc:1428] Setting key c336b0978b58d8f87619029c9923d698
I20260811 11:59:52.607532 30691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edadd11dc3a54ee9be36180cd1f15e60: Bootstrap starting.
I20260811 11:59:52.610306 30691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P edadd11dc3a54ee9be36180cd1f15e60: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:52.611271 30691 log.cc:824] T 00000000000000000000000000000000 P edadd11dc3a54ee9be36180cd1f15e60: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:52.611882 30695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:52.607494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41971 (local address 127.6.143.190:43559)
0811 11:59:52.607987 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:52.608008 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:52.608049 (+ 41us) server_negotiation.cc:407] Connection header received
0811 11:59:52.608915 (+ 866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:52.608940 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:52.609282 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:52.609644 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:52.610367 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:52.610373 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:52.610388 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:52.610409 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:52.610475 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:52.610510 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:52.610631 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:52.610844 (+ 213us) server_negotiation.cc:299] Negotiation successful
0811 11:59:52.610991 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":144,"threads_started":1}
I20260811 11:59:52.613590 30691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edadd11dc3a54ee9be36180cd1f15e60: No bootstrap required, opened a new log
I20260811 11:59:52.614903 6718 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:59:52.618969 30695 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:52.617302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41545 (local address 127.6.143.190:43559)
0811 11:59:52.617447 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:52.617451 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:52.617590 (+ 139us) server_negotiation.cc:407] Connection header received
0811 11:59:52.617670 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:52.617674 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:52.617722 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:52.617828 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:52.618574 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:52.618578 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:52.618580 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:52.618593 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:52.618623 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:52.618638 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:52.618748 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:52.618816 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 11:59:52.618852 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 11:59:52.619071 30691 raft_consensus.cc:348] T 00000000000000000000000000000000 P edadd11dc3a54ee9be36180cd1f15e60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edadd11dc3a54ee9be36180cd1f15e60" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 43559 } }
I20260811 11:59:52.619359 30691 raft_consensus.cc:374] T 00000000000000000000000000000000 P edadd11dc3a54ee9be36180cd1f15e60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:52.619426 30691 raft_consensus.cc:729] T 00000000000000000000000000000000 P edadd11dc3a54ee9be36180cd1f15e60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edadd11dc3a54ee9be36180cd1f15e60, State: Initialized, Role: FOLLOWER
I20260811 11:59:52.619936 30691 consensus_queue.cc:260] T 00000000000000000000000000000000 P edadd11dc3a54ee9be36180cd1f15e60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edadd11dc3a54ee9be36180cd1f15e60" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 43559 } }
I20260811 11:59:52.620138 30691 raft_consensus.cc:388] T 00000000000000000000000000000000 P edadd11dc3a54ee9be36180cd1f15e60 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:59:52.620225 30691 raft_consensus.cc:482] T 00000000000000000000000000000000 P edadd11dc3a54ee9be36180cd1f15e60 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:59:52.620344 30691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P edadd11dc3a54ee9be36180cd1f15e60 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:52.621291 30691 raft_consensus.cc:504] T 00000000000000000000000000000000 P edadd11dc3a54ee9be36180cd1f15e60 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edadd11dc3a54ee9be36180cd1f15e60" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 43559 } }
I20260811 11:59:52.621675 30691 leader_election.cc:302] T 00000000000000000000000000000000 P edadd11dc3a54ee9be36180cd1f15e60 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: edadd11dc3a54ee9be36180cd1f15e60; no voters:
I20260811 11:59:52.622035 30691 leader_election.cc:288] T 00000000000000000000000000000000 P edadd11dc3a54ee9be36180cd1f15e60 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:59:52.622114 30702 raft_consensus.cc:2781] T 00000000000000000000000000000000 P edadd11dc3a54ee9be36180cd1f15e60 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:52.622419 30702 raft_consensus.cc:686] T 00000000000000000000000000000000 P edadd11dc3a54ee9be36180cd1f15e60 [term 1 LEADER]: Becoming Leader. State: Replica: edadd11dc3a54ee9be36180cd1f15e60, State: Running, Role: LEADER
I20260811 11:59:52.622763 30702 consensus_queue.cc:237] T 00000000000000000000000000000000 P edadd11dc3a54ee9be36180cd1f15e60 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edadd11dc3a54ee9be36180cd1f15e60" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 43559 } }
I20260811 11:59:52.623381 30691 sys_catalog.cc:564] T 00000000000000000000000000000000 P edadd11dc3a54ee9be36180cd1f15e60 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:59:52.624679 30703 sys_catalog.cc:455] T 00000000000000000000000000000000 P edadd11dc3a54ee9be36180cd1f15e60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "edadd11dc3a54ee9be36180cd1f15e60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edadd11dc3a54ee9be36180cd1f15e60" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 43559 } } }
I20260811 11:59:52.624830 30703 sys_catalog.cc:458] T 00000000000000000000000000000000 P edadd11dc3a54ee9be36180cd1f15e60 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:52.625141 30704 sys_catalog.cc:455] T 00000000000000000000000000000000 P edadd11dc3a54ee9be36180cd1f15e60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader edadd11dc3a54ee9be36180cd1f15e60. Latest consensus state: current_term: 1 leader_uuid: "edadd11dc3a54ee9be36180cd1f15e60" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "edadd11dc3a54ee9be36180cd1f15e60" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 43559 } } }
I20260811 11:59:52.625866 30704 sys_catalog.cc:458] T 00000000000000000000000000000000 P edadd11dc3a54ee9be36180cd1f15e60 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:52.625891 30711 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:59:52.628664 30711 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:59:52.633615 30711 catalog_manager.cc:1347] Generated new cluster ID: dcbd97f35b7e4035a4f50c3753f218f0
I20260811 11:59:52.633703 30711 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:59:52.649884 30711 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:59:52.650220 30711 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:59:52.656948 30711 catalog_manager.cc:5868] T 00000000000000000000000000000000 P edadd11dc3a54ee9be36180cd1f15e60: Generated new TSK 0
I20260811 11:59:52.657559 30711 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:59:52.662732 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 30616
2026-08-11T11:59:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1444 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:59:52 dist-test-slave-zkpd krb5kdc[30724](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:59:52 dist-test-slave-zkpd krb5kdc[30724](info): set up 2 sockets
Aug 11 11:59:52 dist-test-slave-zkpd krb5kdc[30724](info): commencing operation
krb5kdc: starting...
W20260811 11:59:53.746883 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.060s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:59:53 dist-test-slave-zkpd krb5kdc[30724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:59:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:59:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
I20260811 11:59:53.870325 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:38303
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:43179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:38303
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:53.981022 30739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:53.981249 30739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:53.981290 30739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:53.985080 30739 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:59:53.985128 30739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:53.985146 30739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:53.985162 30739 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:59:53.985177 30739 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:59:53.990253 30739 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:43179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:38303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:38303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:53.991115 30739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:53.992033 30739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:54.000164 30748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:54.000170 30745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:54.000197 30746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:54.000463 30739 server_base.cc:1034] running on GCE node
I20260811 11:59:54.001016 30739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:54.002051 30739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:54.003291 30739 hybrid_clock.cc:648] HybridClock initialized: now 1786449594003274 us; error 25 us; skew 500 ppm
Aug 11 11:59:54 dist-test-slave-zkpd krb5kdc[30724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449594, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:54.006083 30739 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:59:54.006907 30739 webserver.cc:460] Webserver started at http://127.6.143.190:39957/ using document root <none> and password file <none>
I20260811 11:59:54.007463 30739 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:54.007556 30739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:54.007786 30739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:54.009842 30739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "0678aa34cea24a69b5b529f6fe556ec4"
format_stamp: "Formatted at 2026-08-11 11:59:54 on dist-test-slave-zkpd"
server_key: "31591216af327a8cfa5c973f9dea7ce7"
server_key_iv: "0256a2f733098b9bd76548da4ffab968"
server_key_version: "encryptionkey@0"
I20260811 11:59:54.010401 30739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "0678aa34cea24a69b5b529f6fe556ec4"
format_stamp: "Formatted at 2026-08-11 11:59:54 on dist-test-slave-zkpd"
server_key: "31591216af327a8cfa5c973f9dea7ce7"
server_key_iv: "0256a2f733098b9bd76548da4ffab968"
server_key_version: "encryptionkey@0"
I20260811 11:59:54.014108 30739 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 11:59:54.016544 30757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:54.017730 30739 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 11:59:54.017870 30739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "0678aa34cea24a69b5b529f6fe556ec4"
format_stamp: "Formatted at 2026-08-11 11:59:54 on dist-test-slave-zkpd"
server_key: "31591216af327a8cfa5c973f9dea7ce7"
server_key_iv: "0256a2f733098b9bd76548da4ffab968"
server_key_version: "encryptionkey@0"
I20260811 11:59:54.017977 30739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:54.035933 30739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:54.038805 30739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:54.039016 30739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:54.046205 30739 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:38303
I20260811 11:59:54.046257 30812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:38303 every 8 connection(s)
I20260811 11:59:54.047160 30739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:59:54.050081 30813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:54.055612 30813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0678aa34cea24a69b5b529f6fe556ec4: Bootstrap starting.
I20260811 11:59:54.056228 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 30739
I20260811 11:59:54.056388 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:59:54.056684 6718 external_mini_cluster.cc:1428] Setting key 1b73383c851850a6d076bd15b7c056cd
I20260811 11:59:54.058472 30813 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0678aa34cea24a69b5b529f6fe556ec4: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:54.059234 30813 log.cc:824] T 00000000000000000000000000000000 P 0678aa34cea24a69b5b529f6fe556ec4: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:54.061391 30813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0678aa34cea24a69b5b529f6fe556ec4: No bootstrap required, opened a new log
Aug 11 11:59:54 dist-test-slave-zkpd krb5kdc[30724](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449593, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:59:54.065953 30813 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0678aa34cea24a69b5b529f6fe556ec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0678aa34cea24a69b5b529f6fe556ec4" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38303 } }
I20260811 11:59:54.066193 30813 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0678aa34cea24a69b5b529f6fe556ec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:54.066274 30813 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0678aa34cea24a69b5b529f6fe556ec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0678aa34cea24a69b5b529f6fe556ec4, State: Initialized, Role: FOLLOWER
I20260811 11:59:54.066690 30813 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0678aa34cea24a69b5b529f6fe556ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0678aa34cea24a69b5b529f6fe556ec4" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38303 } }
I20260811 11:59:54.066838 30813 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0678aa34cea24a69b5b529f6fe556ec4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:59:54.066913 30813 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0678aa34cea24a69b5b529f6fe556ec4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:59:54.067009 30813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0678aa34cea24a69b5b529f6fe556ec4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:54.067816 30813 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0678aa34cea24a69b5b529f6fe556ec4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0678aa34cea24a69b5b529f6fe556ec4" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38303 } }
I20260811 11:59:54.068209 30813 leader_election.cc:302] T 00000000000000000000000000000000 P 0678aa34cea24a69b5b529f6fe556ec4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0678aa34cea24a69b5b529f6fe556ec4; no voters:
I20260811 11:59:54.068513 30813 leader_election.cc:288] T 00000000000000000000000000000000 P 0678aa34cea24a69b5b529f6fe556ec4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:59:54.068622 30818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0678aa34cea24a69b5b529f6fe556ec4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:54.068897 30818 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0678aa34cea24a69b5b529f6fe556ec4 [term 1 LEADER]: Becoming Leader. State: Replica: 0678aa34cea24a69b5b529f6fe556ec4, State: Running, Role: LEADER
I20260811 11:59:54.069377 30818 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0678aa34cea24a69b5b529f6fe556ec4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0678aa34cea24a69b5b529f6fe556ec4" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38303 } }
I20260811 11:59:54.069871 30813 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0678aa34cea24a69b5b529f6fe556ec4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:59:54.071012 30819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0678aa34cea24a69b5b529f6fe556ec4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0678aa34cea24a69b5b529f6fe556ec4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0678aa34cea24a69b5b529f6fe556ec4" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38303 } } }
I20260811 11:59:54.071138 30819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0678aa34cea24a69b5b529f6fe556ec4 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:54.071527 30820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0678aa34cea24a69b5b529f6fe556ec4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0678aa34cea24a69b5b529f6fe556ec4. Latest consensus state: current_term: 1 leader_uuid: "0678aa34cea24a69b5b529f6fe556ec4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0678aa34cea24a69b5b529f6fe556ec4" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38303 } } }
I20260811 11:59:54.071678 30820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0678aa34cea24a69b5b529f6fe556ec4 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:54.072851 30827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:59:54.073174 30816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:54.058001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36851 (local address 127.6.143.190:38303)
0811 11:59:54.058400 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:54.058416 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:54.058450 (+ 34us) server_negotiation.cc:407] Connection header received
0811 11:59:54.059116 (+ 666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:54.059133 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:54.059507 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:54.059879 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:54.060838 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:54.061850 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:54.063022 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:54.063454 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:54.063562 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:54.066513 (+ 2951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:54.066522 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:54.066538 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:54.066569 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:54.068536 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:54.069256 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:54.069263 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:54.069294 (+ 31us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:54.069449 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:54.069775 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:54.069778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:54.069780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:54.070313 (+ 533us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:54.070516 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:54.071438 (+ 922us) server_negotiation.cc:299] Negotiation successful
0811 11:59:54.071761 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":126,"threads_started":1}
I20260811 11:59:54.075867 6718 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:59:54.076056 30827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:59:54.082500 30827 catalog_manager.cc:1347] Generated new cluster ID: d525f9c2136a496fb5aba110e0a3f2b8
I20260811 11:59:54.082617 30827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:59:54.086143 30816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:54.077871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58553 (local address 127.6.143.190:38303)
0811 11:59:54.077966 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:54.077970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:54.078256 (+ 286us) server_negotiation.cc:407] Connection header received
0811 11:59:54.078390 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:54.078393 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:54.078434 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:54.078522 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:54.079144 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:54.079638 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:54.080790 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:54.081134 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:54.081233 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:54.082904 (+ 1671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:54.082908 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:54.082910 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:54.082935 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:54.084437 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:54.084865 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:54.084869 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:54.084871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:54.084954 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:54.085235 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:54.085239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:54.085240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:54.085421 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:54.085578 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:54.085815 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 11:59:54.085972 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260811 11:59:54.090554 30827 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:59:54.091683 30827 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:59:54.097038 30827 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0678aa34cea24a69b5b529f6fe556ec4: Generated new TSK 0
I20260811 11:59:54.097693 30827 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:59:54.105207 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 30739
2026-08-11T11:59:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1441 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:59:54 dist-test-slave-zkpd krb5kdc[30849](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:59:54 dist-test-slave-zkpd krb5kdc[30849](info): set up 2 sockets
Aug 11 11:59:54 dist-test-slave-zkpd krb5kdc[30849](info): commencing operation
krb5kdc: starting...
W20260811 11:59:55.197770 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:59:55 dist-test-slave-zkpd krb5kdc[30849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449595, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:59:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:59:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
I20260811 11:59:55.324734 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:33509
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:40623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:33509
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:55.437422 30862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:55.437708 30862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:55.437784 30862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:55.443956 30862 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:59:55.444034 30862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:55.444067 30862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:55.444149 30862 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:59:55.444216 30862 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:59:55.451663 30862 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:40623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:33509
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:33509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:55.452713 30862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:55.453723 30862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:55.463179 30869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:55.463197 30872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:55.463373 30862 server_base.cc:1034] running on GCE node
W20260811 11:59:55.463425 30870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:55.464330 30862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:55.465761 30862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:55.466902 30862 hybrid_clock.cc:648] HybridClock initialized: now 1786449595466883 us; error 26 us; skew 500 ppm
Aug 11 11:59:55 dist-test-slave-zkpd krb5kdc[30849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449595, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:55.470795 30862 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:59:55.471712 30862 webserver.cc:460] Webserver started at http://127.6.143.190:41623/ using document root <none> and password file <none>
I20260811 11:59:55.472414 30862 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:55.472481 30862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:55.472711 30862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:55.474732 30862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "ce46f8ed3d114d3284480f1fbf1d3c5f"
format_stamp: "Formatted at 2026-08-11 11:59:55 on dist-test-slave-zkpd"
server_key: "5005b3125a32555986b753e28374a598"
server_key_iv: "be62af4671a1e9910fedf9694e601886"
server_key_version: "encryptionkey@0"
I20260811 11:59:55.475301 30862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "ce46f8ed3d114d3284480f1fbf1d3c5f"
format_stamp: "Formatted at 2026-08-11 11:59:55 on dist-test-slave-zkpd"
server_key: "5005b3125a32555986b753e28374a598"
server_key_iv: "be62af4671a1e9910fedf9694e601886"
server_key_version: "encryptionkey@0"
I20260811 11:59:55.478947 30862 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260811 11:59:55.481431 30881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:55.482671 30862 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 11:59:55.482826 30862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "ce46f8ed3d114d3284480f1fbf1d3c5f"
format_stamp: "Formatted at 2026-08-11 11:59:55 on dist-test-slave-zkpd"
server_key: "5005b3125a32555986b753e28374a598"
server_key_iv: "be62af4671a1e9910fedf9694e601886"
server_key_version: "encryptionkey@0"
I20260811 11:59:55.482941 30862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:55.510478 30862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:55.513957 30862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:55.514169 30862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:55.521404 30862 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:33509
I20260811 11:59:55.521411 30938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:33509 every 8 connection(s)
I20260811 11:59:55.522320 30862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:59:55.525520 30939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:55.531594 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 30862
I20260811 11:59:55.531708 30939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce46f8ed3d114d3284480f1fbf1d3c5f: Bootstrap starting.
I20260811 11:59:55.531787 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:59:55.532145 6718 external_mini_cluster.cc:1428] Setting key 7a2f993870187f73ac9d79c8a95e8fb2
I20260811 11:59:55.534583 30939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ce46f8ed3d114d3284480f1fbf1d3c5f: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:55.535390 30939 log.cc:824] T 00000000000000000000000000000000 P ce46f8ed3d114d3284480f1fbf1d3c5f: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:55.537493 30939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ce46f8ed3d114d3284480f1fbf1d3c5f: No bootstrap required, opened a new log
I20260811 11:59:55.541095 30942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:55.533591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35241 (local address 127.6.143.190:33509)
0811 11:59:55.534076 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:55.534092 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:55.534127 (+ 35us) server_negotiation.cc:407] Connection header received
0811 11:59:55.534813 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:55.534832 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:55.535176 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:55.535507 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:55.535955 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:55.536953 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:55.537917 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:55.538399 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:55.538572 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:55.539438 (+ 866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:55.539443 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:55.539456 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:55.539825 (+ 369us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:55.539875 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:55.539957 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:55.540071 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:55.540188 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 11:59:55.540395 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":344,"thread_start_us":141,"threads_started":1}
I20260811 11:59:55.541567 30939 raft_consensus.cc:348] T 00000000000000000000000000000000 P ce46f8ed3d114d3284480f1fbf1d3c5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce46f8ed3d114d3284480f1fbf1d3c5f" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 33509 } }
I20260811 11:59:55.541848 30939 raft_consensus.cc:374] T 00000000000000000000000000000000 P ce46f8ed3d114d3284480f1fbf1d3c5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:55.541923 30939 raft_consensus.cc:729] T 00000000000000000000000000000000 P ce46f8ed3d114d3284480f1fbf1d3c5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce46f8ed3d114d3284480f1fbf1d3c5f, State: Initialized, Role: FOLLOWER
I20260811 11:59:55.542405 30939 consensus_queue.cc:260] T 00000000000000000000000000000000 P ce46f8ed3d114d3284480f1fbf1d3c5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce46f8ed3d114d3284480f1fbf1d3c5f" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 33509 } }
I20260811 11:59:55.542580 30939 raft_consensus.cc:388] T 00000000000000000000000000000000 P ce46f8ed3d114d3284480f1fbf1d3c5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:59:55.542673 30939 raft_consensus.cc:482] T 00000000000000000000000000000000 P ce46f8ed3d114d3284480f1fbf1d3c5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:59:55.542812 30939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ce46f8ed3d114d3284480f1fbf1d3c5f [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:55.544034 30939 raft_consensus.cc:504] T 00000000000000000000000000000000 P ce46f8ed3d114d3284480f1fbf1d3c5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce46f8ed3d114d3284480f1fbf1d3c5f" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 33509 } }
I20260811 11:59:55.544201 6718 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:59:55.544435 30939 leader_election.cc:302] T 00000000000000000000000000000000 P ce46f8ed3d114d3284480f1fbf1d3c5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ce46f8ed3d114d3284480f1fbf1d3c5f; no voters:
I20260811 11:59:55.544770 30939 leader_election.cc:288] T 00000000000000000000000000000000 P ce46f8ed3d114d3284480f1fbf1d3c5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:59:55.544876 30944 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ce46f8ed3d114d3284480f1fbf1d3c5f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:55.545146 30944 raft_consensus.cc:686] T 00000000000000000000000000000000 P ce46f8ed3d114d3284480f1fbf1d3c5f [term 1 LEADER]: Becoming Leader. State: Replica: ce46f8ed3d114d3284480f1fbf1d3c5f, State: Running, Role: LEADER
I20260811 11:59:55.545522 30944 consensus_queue.cc:237] T 00000000000000000000000000000000 P ce46f8ed3d114d3284480f1fbf1d3c5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce46f8ed3d114d3284480f1fbf1d3c5f" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 33509 } }
I20260811 11:59:55.546128 30939 sys_catalog.cc:564] T 00000000000000000000000000000000 P ce46f8ed3d114d3284480f1fbf1d3c5f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:59:55.546815 30946 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce46f8ed3d114d3284480f1fbf1d3c5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ce46f8ed3d114d3284480f1fbf1d3c5f. Latest consensus state: current_term: 1 leader_uuid: "ce46f8ed3d114d3284480f1fbf1d3c5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce46f8ed3d114d3284480f1fbf1d3c5f" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 33509 } } }
I20260811 11:59:55.547700 30946 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce46f8ed3d114d3284480f1fbf1d3c5f [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:55.547567 30945 sys_catalog.cc:455] T 00000000000000000000000000000000 P ce46f8ed3d114d3284480f1fbf1d3c5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ce46f8ed3d114d3284480f1fbf1d3c5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ce46f8ed3d114d3284480f1fbf1d3c5f" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 33509 } } }
I20260811 11:59:55.548605 30945 sys_catalog.cc:458] T 00000000000000000000000000000000 P ce46f8ed3d114d3284480f1fbf1d3c5f [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:55.548732 30961 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:59:55.551599 30942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:55.547059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33613 (local address 127.6.143.190:33509)
0811 11:59:55.547162 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:55.547166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:55.547733 (+ 567us) server_negotiation.cc:407] Connection header received
0811 11:59:55.547942 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:55.547945 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:55.547990 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:55.548046 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:55.548528 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:55.549156 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:55.549946 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:55.550232 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:55.550351 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:55.551155 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:55.551159 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:55.551161 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:55.551173 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:55.551207 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:55.551218 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:55.551299 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:55.551391 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 11:59:55.551445 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 11:59:55.552654 30961 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:59:55.558941 30961 catalog_manager.cc:1347] Generated new cluster ID: 6a9cfa530b934a9db45eda3ed42e44c6
I20260811 11:59:55.559011 30961 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:59:55.592079 30961 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:59:55.593044 30961 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:59:55.599459 30961 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ce46f8ed3d114d3284480f1fbf1d3c5f: Generated new TSK 0
I20260811 11:59:55.600159 30961 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:59:55.607698 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 30862
2026-08-11T11:59:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1502 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:59:55 dist-test-slave-zkpd krb5kdc[30977](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:59:55 dist-test-slave-zkpd krb5kdc[30977](info): set up 2 sockets
Aug 11 11:59:55 dist-test-slave-zkpd krb5kdc[30977](info): commencing operation
krb5kdc: starting...
W20260811 11:59:56.689991 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.059s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:59:56 dist-test-slave-zkpd krb5kdc[30977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449596, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:59:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:59:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
I20260811 11:59:56.812728 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:42849
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:35303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:42849
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:56.925168 30991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:56.925405 30991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:56.925444 30991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:56.929324 30991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:59:56.929373 30991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:56.929390 30991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:56.929406 30991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:59:56.929420 30991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:59:56.934463 30991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:35303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:42849
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:42849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:56.935418 30991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:56.936429 30991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:56.944756 30998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:56.944742 30997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:56.944738 31000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:56.945112 30991 server_base.cc:1034] running on GCE node
I20260811 11:59:56.945674 30991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:56.946766 30991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:56.947973 30991 hybrid_clock.cc:648] HybridClock initialized: now 1786449596947954 us; error 36 us; skew 500 ppm
Aug 11 11:59:56 dist-test-slave-zkpd krb5kdc[30977](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449596, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:56.950650 30991 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:59:56.951561 30991 webserver.cc:460] Webserver started at http://127.6.143.190:41123/ using document root <none> and password file <none>
I20260811 11:59:56.952184 30991 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:56.952298 30991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:56.952533 30991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:56.954541 30991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "d47a75831682456190f52255967773ce"
format_stamp: "Formatted at 2026-08-11 11:59:56 on dist-test-slave-zkpd"
server_key: "3edb75e6c134ed8821eac012a5dc96ac"
server_key_iv: "0e6138d1d7f225062de6ea27e97b568b"
server_key_version: "encryptionkey@0"
I20260811 11:59:56.955120 30991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "d47a75831682456190f52255967773ce"
format_stamp: "Formatted at 2026-08-11 11:59:56 on dist-test-slave-zkpd"
server_key: "3edb75e6c134ed8821eac012a5dc96ac"
server_key_iv: "0e6138d1d7f225062de6ea27e97b568b"
server_key_version: "encryptionkey@0"
I20260811 11:59:56.958820 30991 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 11:59:56.961230 31010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:56.962421 30991 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 11:59:56.962567 30991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "d47a75831682456190f52255967773ce"
format_stamp: "Formatted at 2026-08-11 11:59:56 on dist-test-slave-zkpd"
server_key: "3edb75e6c134ed8821eac012a5dc96ac"
server_key_iv: "0e6138d1d7f225062de6ea27e97b568b"
server_key_version: "encryptionkey@0"
I20260811 11:59:56.962677 30991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:56.967303 30991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:56.970114 30991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:56.970340 30991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:56.977458 30991 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:42849
I20260811 11:59:56.977492 31064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:42849 every 8 connection(s)
I20260811 11:59:56.978358 30991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:59:56.978672 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 30991
I20260811 11:59:56.978843 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:59:56.979146 6718 external_mini_cluster.cc:1428] Setting key 14f15fcceb1ec7a20bc0ea388ff6bc86
I20260811 11:59:56.981316 31065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:56.984481 31068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:56.980449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42645 (local address 127.6.143.190:42849)
0811 11:59:56.980856 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:56.980878 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:56.981112 (+ 234us) server_negotiation.cc:407] Connection header received
0811 11:59:56.981929 (+ 817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:56.981947 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:56.982222 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:56.982553 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:56.983267 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:56.983272 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:56.983296 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:56.983306 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:56.983353 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:56.983378 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:56.983532 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:56.983604 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 11:59:56.983750 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":140,"threads_started":1}
I20260811 11:59:56.987246 6718 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:59:56.987645 31065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d47a75831682456190f52255967773ce: Bootstrap starting.
I20260811 11:59:56.990091 31065 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d47a75831682456190f52255967773ce: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:56.990978 31068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:56.989433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53387 (local address 127.6.143.190:42849)
0811 11:59:56.989539 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:56.989544 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:56.989691 (+ 147us) server_negotiation.cc:407] Connection header received
0811 11:59:56.989803 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:56.989805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:56.989831 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:56.989894 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:56.990588 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:56.990591 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:56.990592 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 11:59:56.990603 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:56.990626 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 11:59:56.990636 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:56.990704 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:56.990818 (+ 114us) server_negotiation.cc:299] Negotiation successful
0811 11:59:56.990848 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 11:59:56.991070 31065 log.cc:824] T 00000000000000000000000000000000 P d47a75831682456190f52255967773ce: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:56.993026 31065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d47a75831682456190f52255967773ce: No bootstrap required, opened a new log
I20260811 11:59:56.996819 31065 raft_consensus.cc:348] T 00000000000000000000000000000000 P d47a75831682456190f52255967773ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d47a75831682456190f52255967773ce" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 42849 } }
I20260811 11:59:56.997026 31065 raft_consensus.cc:374] T 00000000000000000000000000000000 P d47a75831682456190f52255967773ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 11:59:56.997114 31065 raft_consensus.cc:729] T 00000000000000000000000000000000 P d47a75831682456190f52255967773ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d47a75831682456190f52255967773ce, State: Initialized, Role: FOLLOWER
I20260811 11:59:56.997572 31065 consensus_queue.cc:260] T 00000000000000000000000000000000 P d47a75831682456190f52255967773ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d47a75831682456190f52255967773ce" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 42849 } }
I20260811 11:59:56.997717 31065 raft_consensus.cc:388] T 00000000000000000000000000000000 P d47a75831682456190f52255967773ce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 11:59:56.997793 31065 raft_consensus.cc:482] T 00000000000000000000000000000000 P d47a75831682456190f52255967773ce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 11:59:56.997891 31065 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d47a75831682456190f52255967773ce [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:56.998780 31065 raft_consensus.cc:504] T 00000000000000000000000000000000 P d47a75831682456190f52255967773ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d47a75831682456190f52255967773ce" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 42849 } }
I20260811 11:59:56.999117 31065 leader_election.cc:302] T 00000000000000000000000000000000 P d47a75831682456190f52255967773ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d47a75831682456190f52255967773ce; no voters:
I20260811 11:59:56.999423 31065 leader_election.cc:288] T 00000000000000000000000000000000 P d47a75831682456190f52255967773ce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 11:59:56.999542 31075 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d47a75831682456190f52255967773ce [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:56.999857 31075 raft_consensus.cc:686] T 00000000000000000000000000000000 P d47a75831682456190f52255967773ce [term 1 LEADER]: Becoming Leader. State: Replica: d47a75831682456190f52255967773ce, State: Running, Role: LEADER
I20260811 11:59:57.000218 31075 consensus_queue.cc:237] T 00000000000000000000000000000000 P d47a75831682456190f52255967773ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d47a75831682456190f52255967773ce" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 42849 } }
I20260811 11:59:57.000708 31065 sys_catalog.cc:564] T 00000000000000000000000000000000 P d47a75831682456190f52255967773ce [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:59:57.001338 31075 sys_catalog.cc:455] T 00000000000000000000000000000000 P d47a75831682456190f52255967773ce [sys.catalog]: SysCatalogTable state changed. Reason: New leader d47a75831682456190f52255967773ce. Latest consensus state: current_term: 1 leader_uuid: "d47a75831682456190f52255967773ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d47a75831682456190f52255967773ce" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 42849 } } }
I20260811 11:59:57.001529 31075 sys_catalog.cc:458] T 00000000000000000000000000000000 P d47a75831682456190f52255967773ce [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:57.001761 31077 sys_catalog.cc:455] T 00000000000000000000000000000000 P d47a75831682456190f52255967773ce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d47a75831682456190f52255967773ce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d47a75831682456190f52255967773ce" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 42849 } } }
I20260811 11:59:57.001907 31077 sys_catalog.cc:458] T 00000000000000000000000000000000 P d47a75831682456190f52255967773ce [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:57.002010 31084 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:59:57.004951 31084 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:59:57.010577 31084 catalog_manager.cc:1347] Generated new cluster ID: 6107cecc2fda4aab92704c017992ba4c
I20260811 11:59:57.010694 31084 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:59:57.031193 31084 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:59:57.031553 31084 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:59:57.039269 31084 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d47a75831682456190f52255967773ce: Generated new TSK 0
I20260811 11:59:57.039951 31084 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 11:59:57.050053 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 30991
2026-08-11T11:59:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1441 ms)
[----------] 18 tests from AuthTokenIssuingTest (26363 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 11:59:57 dist-test-slave-zkpd krb5kdc[31097](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 11:59:57 dist-test-slave-zkpd krb5kdc[31097](info): set up 2 sockets
Aug 11 11:59:57 dist-test-slave-zkpd krb5kdc[31097](info): commencing operation
krb5kdc: starting...
W20260811 11:59:58.138846 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 11:59:58 dist-test-slave-zkpd krb5kdc[31097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T11:59:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T11:59:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
I20260811 11:59:58.270991 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:38021
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:39687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:38021,127.6.143.189:36167,127.6.143.188:37747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 11:59:58.383351 31110 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 11:59:58.383661 31110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:58.383729 31110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:58.383812 31110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:58.387775 31110 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:59:58.387856 31110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:58.387889 31110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:58.387945 31110 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:59:58.387981 31110 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:59:58.393024 31110 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:39687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:38021,127.6.143.189:36167,127.6.143.188:37747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:38021
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:58.394057 31110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:58.395064 31110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 11:59:58.403473 31110 server_base.cc:1034] running on GCE node
W20260811 11:59:58.403430 31117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:58.403463 31116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:58.403429 31119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:58.404217 31110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:58.405270 31110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:58.406458 31110 hybrid_clock.cc:648] HybridClock initialized: now 1786449598406436 us; error 45 us; skew 500 ppm
Aug 11 11:59:58 dist-test-slave-zkpd krb5kdc[31097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449598, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:58.413267 31110 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 11:59:58.414170 31110 webserver.cc:460] Webserver started at http://127.6.143.190:42003/ using document root <none> and password file <none>
I20260811 11:59:58.414739 31110 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:58.414834 31110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:58.415076 31110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:58.417209 31110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "5bf3f8e00a484315b3ed43089ac18ee3"
format_stamp: "Formatted at 2026-08-11 11:59:58 on dist-test-slave-zkpd"
server_key: "283eb8ee6c31a55aeb6f80a8ee4f2169"
server_key_iv: "11ba2b9114370d4d38c7e73779b718b9"
server_key_version: "encryptionkey@0"
I20260811 11:59:58.417795 31110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "5bf3f8e00a484315b3ed43089ac18ee3"
format_stamp: "Formatted at 2026-08-11 11:59:58 on dist-test-slave-zkpd"
server_key: "283eb8ee6c31a55aeb6f80a8ee4f2169"
server_key_iv: "11ba2b9114370d4d38c7e73779b718b9"
server_key_version: "encryptionkey@0"
I20260811 11:59:58.421587 31110 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 11:59:58.424100 31126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:58.425334 31110 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:59:58.425482 31110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "5bf3f8e00a484315b3ed43089ac18ee3"
format_stamp: "Formatted at 2026-08-11 11:59:58 on dist-test-slave-zkpd"
server_key: "283eb8ee6c31a55aeb6f80a8ee4f2169"
server_key_iv: "11ba2b9114370d4d38c7e73779b718b9"
server_key_version: "encryptionkey@0"
I20260811 11:59:58.425593 31110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:58.434705 31110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:58.437544 31110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:58.437748 31110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:58.444918 31110 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:38021
I20260811 11:59:58.445225 31184 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:38021 every 8 connection(s)
I20260811 11:59:58.445886 31110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 11:59:58.447149 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 31110
I20260811 11:59:58.447292 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 11:59:58.447567 6718 external_mini_cluster.cc:1428] Setting key 021492c4461b8f70c145aa82c4650b43
I20260811 11:59:58.448817 31185 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:58.452507 31185 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38021 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.143.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.189@KRBTEST.COM" created.
Aug 11 11:59:58 dist-test-slave-zkpd krb5kdc[31097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449598, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:59:58.468022 31189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.454710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:38021 (local address 127.0.0.1:41786)
0811 11:59:58.455276 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:58.455311 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:58.456146 (+ 835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:58.456840 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:58.456845 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:58.457280 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:59:58.457519 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:58.457533 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.458755 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.458759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:58.459780 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:58.459796 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.460532 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.460540 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:58.460706 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:58.461333 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:59:58.461349 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:59:58.462750 (+ 1401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:59:58.465221 (+ 2471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:58.465230 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:58.465237 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:58.465465 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:58.465869 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:58.465871 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:58.465872 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:58.466002 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:58.466724 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:59:58.466728 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:59:58.466841 (+ 113us) client_negotiation.cc:769] Sending connection context
0811 11:59:58.466985 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 11:59:58.467215 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":158,"threads_started":1}
I20260811 11:59:58.468022 31188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.454710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41786 (local address 127.6.143.190:38021)
0811 11:59:58.455128 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:58.455147 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:58.455414 (+ 267us) server_negotiation.cc:407] Connection header received
0811 11:59:58.456479 (+ 1065us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:58.456483 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:58.456656 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:58.456841 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:58.457676 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.458629 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.459978 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.460361 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.460489 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:58.462911 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:58.462920 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:58.462935 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:58.462964 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:58.465107 (+ 2143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:58.465583 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:58.465592 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:58.465594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:58.465754 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:58.466103 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:58.466108 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:58.466109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:58.466607 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:58.466768 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:58.467115 (+ 347us) server_negotiation.cc:299] Negotiation successful
0811 11:59:58.467289 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":145,"threads_started":1}
I20260811 11:59:58.470507 31185 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.189" port: 36167 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:59:58.471202 31189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.470979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:36167 (local address 127.0.0.1:50014)
0811 11:59:58.471087 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:58.471153 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.189:36167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 11:59:58.471415 31129 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.143.189:36167: connect: Connection refused (error 111)
I20260811 11:59:58.472409 31189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.472173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:36167 (local address 127.0.0.1:50016)
0811 11:59:58.472306 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:58.472360 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.189:36167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 11:59:58.472620 31185 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.143.189:36167: Network error: Client connection negotiation failed: client connection to 127.6.143.189:36167: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.189.keytab.
Entry for principal kudu/127.6.143.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.189.keytab.
I20260811 11:59:58.474676 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.189:36167
--webserver_interface=127.6.143.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.189.keytab
--principal=kudu/127.6.143.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:39687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:38021,127.6.143.189:36167,127.6.143.188:37747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:59:58.521915 31185 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.143.189" port: 36167 } attempt: 1
I20260811 11:59:58.522811 31189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.522518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:36167 (local address 127.0.0.1:50028)
0811 11:59:58.522674 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:58.522737 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.189:36167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 11:59:58.523588 31189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.523343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:36167 (local address 127.0.0.1:50034)
0811 11:59:58.523456 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:58.523518 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.189:36167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 11:59:58.523715 31185 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.143.189:36167: Network error: Client connection negotiation failed: client connection to 127.6.143.189:36167: connect: Connection refused (error 111)
W20260811 11:59:58.585601 31193 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 11:59:58.585901 31193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:58.586000 31193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:58.586077 31193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 11:59:58.589999 31193 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:59:58.590080 31193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:58.590118 31193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:58.590155 31193 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:59:58.590190 31193 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:59:58.591965 31185 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.143.189" port: 36167 } attempt: 2
I20260811 11:59:58.592775 31189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.592482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:36167 (local address 127.0.0.1:50046)
0811 11:59:58.592638 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:58.592704 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.189:36167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 11:59:58.593465 31189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.593258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:36167 (local address 127.0.0.1:50048)
0811 11:59:58.593364 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:58.593415 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.189:36167: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 11:59:58.593580 31185 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.143.189:36167: Network error: Client connection negotiation failed: client connection to 127.6.143.189:36167: connect: Connection refused (error 111)
I20260811 11:59:58.595479 31193 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:39687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:38021,127.6.143.189:36167,127.6.143.188:37747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.189.keytab
--principal=kudu/127.6.143.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.189:36167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.143.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:58.596524 31193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:58.597491 31193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:58.605683 31198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:58.605667 31199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:58.605659 31201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:58.605729 31193 server_base.cc:1034] running on GCE node
I20260811 11:59:58.606428 31193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:58.607438 31193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:58.608595 31193 hybrid_clock.cc:648] HybridClock initialized: now 1786449598608560 us; error 44 us; skew 500 ppm
Aug 11 11:59:58 dist-test-slave-zkpd krb5kdc[31097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449598, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:58.611265 31193 init.cc:377] Logged in from keytab as kudu/127.6.143.189@KRBTEST.COM (short username kudu)
I20260811 11:59:58.612185 31193 webserver.cc:460] Webserver started at http://127.6.143.189:45707/ using document root <none> and password file <none>
I20260811 11:59:58.612730 31193 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:58.612789 31193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:58.613034 31193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:58.614971 31193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-1/data/instance:
uuid: "57ecbe71f3b04dd592049175623572d3"
format_stamp: "Formatted at 2026-08-11 11:59:58 on dist-test-slave-zkpd"
server_key: "7d26aafdc4742a2f408957e0ae2b1156"
server_key_iv: "cc119e2bdd3d7bff4b5e29046701a22c"
server_key_version: "encryptionkey@0"
I20260811 11:59:58.615514 31193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-1/wal/instance:
uuid: "57ecbe71f3b04dd592049175623572d3"
format_stamp: "Formatted at 2026-08-11 11:59:58 on dist-test-slave-zkpd"
server_key: "7d26aafdc4742a2f408957e0ae2b1156"
server_key_iv: "cc119e2bdd3d7bff4b5e29046701a22c"
server_key_version: "encryptionkey@0"
I20260811 11:59:58.619048 31193 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260811 11:59:58.621454 31208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:58.622643 31193 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 11:59:58.622802 31193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-1/wal
uuid: "57ecbe71f3b04dd592049175623572d3"
format_stamp: "Formatted at 2026-08-11 11:59:58 on dist-test-slave-zkpd"
server_key: "7d26aafdc4742a2f408957e0ae2b1156"
server_key_iv: "cc119e2bdd3d7bff4b5e29046701a22c"
server_key_version: "encryptionkey@0"
I20260811 11:59:58.622913 31193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:58.662781 31193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:58.666245 31193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:58.666481 31193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:58.673950 31193 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.189:36167
I20260811 11:59:58.674269 31264 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.189:36167 every 8 connection(s)
I20260811 11:59:58.674944 31193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-1/data/info.pb
I20260811 11:59:58.677847 31265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:58.681843 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 31193
I20260811 11:59:58.681981 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-1/wal/instance
I20260811 11:59:58.681761 31265 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38021 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:59:58.682266 6718 external_mini_cluster.cc:1428] Setting key 570c80d7ee5e00056aa37dca84013b7c
I20260811 11:59:58.684764 31185 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.143.189" port: 36167 } attempt: 3
Aug 11 11:59:58 dist-test-slave-zkpd krb5kdc[31097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449598, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for kudu/127.6.143.189@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.143.188@KRBTEST.COM; defaulting to no policy
Aug 11 11:59:58 dist-test-slave-zkpd krb5kdc[31097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449598, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.189@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.188@KRBTEST.COM" created.
I20260811 11:59:58.696697 31189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.685212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:36167 (local address 127.0.0.1:50062)
0811 11:59:58.685392 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:58.685407 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:58.685495 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:58.686576 (+ 1081us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:58.686579 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:58.686747 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:59:58.686847 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:58.686853 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.688819 (+ 1966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.688823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:58.689376 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:58.689385 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.690152 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.690156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:58.690296 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:58.690821 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:59:58.690836 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:59:58.692520 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:59:58.694635 (+ 2115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:58.694641 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:58.694643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:58.694851 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:58.695194 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:58.695198 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:58.695200 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:58.695313 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:58.696096 (+ 783us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:59:58.696099 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:59:58.696221 (+ 122us) client_negotiation.cc:769] Sending connection context
0811 11:59:58.696358 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 11:59:58.696495 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"spinlock_wait_cycles":4992}
I20260811 11:59:58.697827 31188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.684032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41794 (local address 127.6.143.190:38021)
0811 11:59:58.684201 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:58.684206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:58.685075 (+ 869us) server_negotiation.cc:407] Connection header received
0811 11:59:58.686290 (+ 1215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:58.686293 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:58.686343 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:58.686447 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:58.687751 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.688524 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.690052 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.690405 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.690534 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:58.693440 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:58.693447 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:58.693453 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:58.693485 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:58.695093 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:58.695754 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:58.695759 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:58.695760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:58.695847 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:58.696239 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:58.696243 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:58.696244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:58.696449 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:58.696517 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:58.697566 (+ 1049us) server_negotiation.cc:299] Negotiation successful
0811 11:59:58.697701 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 11:59:58.698105 31268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.684436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:38021 (local address 127.0.0.1:41794)
0811 11:59:58.684913 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:58.684966 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:58.686060 (+ 1094us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:58.686508 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:58.686520 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:58.686997 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:59:58.687581 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:58.687601 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.688652 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.688658 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:58.689911 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:58.689926 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.690709 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.690715 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:58.690839 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:58.691368 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:59:58.691384 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:59:58.693244 (+ 1860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:59:58.695280 (+ 2036us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:58.695293 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:58.695302 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:58.695637 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:58.695982 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:58.695985 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:58.695987 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:58.696102 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:58.696604 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:59:58.696610 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:59:58.696734 (+ 124us) client_negotiation.cc:769] Sending connection context
0811 11:59:58.696887 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 11:59:58.697092 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":30,"thread_start_us":129,"threads_started":1}
I20260811 11:59:58.698108 31269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.685575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50062 (local address 127.6.143.189:36167)
0811 11:59:58.685921 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:58.685938 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:58.685983 (+ 45us) server_negotiation.cc:407] Connection header received
0811 11:59:58.686152 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:58.686166 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:58.686393 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:58.686994 (+ 601us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:58.687381 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.688713 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.689508 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.689969 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.690185 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:58.692686 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:58.692694 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:58.692710 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:58.692738 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:58.694420 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:58.694972 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:58.694978 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:58.694984 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:58.695093 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:58.695437 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:58.695439 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:58.695440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:58.695978 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:58.696118 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:58.696532 (+ 414us) server_negotiation.cc:299] Negotiation successful
0811 11:59:58.697032 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":216,"thread_start_us":126,"threads_started":1}
I20260811 11:59:58.699462 31265 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.189" port: 36167 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:59:58.700197 31185 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.188" port: 37747 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:59:58.700973 31189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.700766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:37747 (local address 127.0.0.1:34984)
0811 11:59:58.700858 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:58.700908 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:37747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260811 11:59:58.701656 31189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.701423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:37747 (local address 127.0.0.1:35000)
0811 11:59:58.701556 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:58.701604 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:37747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260811 11:59:58.701829 31185 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.143.188:37747: Network error: Client connection negotiation failed: client connection to 127.6.143.188:37747: connect: Connection refused (error 111)
Aug 11 11:59:58 dist-test-slave-zkpd krb5kdc[31097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449598, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.189@KRBTEST.COM for kudu/127.6.143.189@KRBTEST.COM
I20260811 11:59:58.708458 31268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.699925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:36167 (local address 127.0.0.1:50074)
0811 11:59:58.700141 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:58.700154 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:58.700233 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:58.700953 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:58.700958 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:58.701138 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:59:58.701243 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:58.701250 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.702370 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.702373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:58.702870 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:58.702878 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.703505 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.703507 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:58.703612 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:58.704112 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:59:58.704125 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:59:58.705432 (+ 1307us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:59:58.707024 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:58.707031 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:58.707033 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:58.707221 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:58.707500 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:58.707505 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:58.707507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:58.707586 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:58.707954 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:59:58.707957 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:59:58.708048 (+ 91us) client_negotiation.cc:769] Sending connection context
0811 11:59:58.708165 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 11:59:58.708293 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":2}
I20260811 11:59:58.708472 31269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.700053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50074 (local address 127.6.143.189:36167)
0811 11:59:58.700641 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:58.700646 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:58.700662 (+ 16us) server_negotiation.cc:407] Connection header received
0811 11:59:58.700700 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:58.700702 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:58.700751 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:58.700841 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:58.701367 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.702234 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.702994 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.703377 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.703500 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:58.705557 (+ 2057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:58.705560 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:58.705562 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:58.705583 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:58.706912 (+ 1329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:58.707318 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:58.707323 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:58.707325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:58.707409 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:58.707675 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:58.707679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:58.707681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:58.707839 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:58.707961 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:58.708210 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 11:59:58.708346 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":494}
I20260811 11:59:58.709355 31265 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.188" port: 37747 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:59:58.710019 31268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.709752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:37747 (local address 127.0.0.1:35014)
0811 11:59:58.709859 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:58.709967 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:37747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 11:59:58.710678 31211 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.143.188:37747: connect: Connection refused (error 111)
I20260811 11:59:58.711576 31268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.711354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:37747 (local address 127.0.0.1:35024)
0811 11:59:58.711491 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:58.711531 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:37747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 11:59:58.711776 31265 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.143.188:37747: Network error: Client connection negotiation failed: client connection to 127.6.143.188:37747: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.188.keytab.
Entry for principal kudu/127.6.143.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.188.keytab.
I20260811 11:59:58.713151 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.188:37747
--webserver_interface=127.6.143.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.188.keytab
--principal=kudu/127.6.143.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:39687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:38021,127.6.143.189:36167,127.6.143.188:37747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 11:59:58.733000 31185 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.143.188" port: 37747 } attempt: 1
I20260811 11:59:58.733875 31189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.733521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:37747 (local address 127.0.0.1:35028)
0811 11:59:58.733731 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:58.733795 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:37747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260811 11:59:58.734727 31189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.734430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:37747 (local address 127.0.0.1:35042)
0811 11:59:58.734549 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:58.734609 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:37747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 11:59:58.734928 31185 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.143.188:37747: Network error: Client connection negotiation failed: client connection to 127.6.143.188:37747: connect: Connection refused (error 111)
I20260811 11:59:58.761073 31265 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.143.188" port: 37747 } attempt: 1
I20260811 11:59:58.761950 31268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.761606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:37747 (local address 127.0.0.1:35054)
0811 11:59:58.761785 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:58.761858 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:37747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260811 11:59:58.762745 31268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.762504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:37747 (local address 127.0.0.1:35058)
0811 11:59:58.762618 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:58.762673 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:37747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 11:59:58.762895 31265 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.143.188:37747: Network error: Client connection negotiation failed: client connection to 127.6.143.188:37747: connect: Connection refused (error 111)
I20260811 11:59:58.810112 31185 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.143.188" port: 37747 } attempt: 2
I20260811 11:59:58.810990 31189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.810691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:37747 (local address 127.0.0.1:35064)
0811 11:59:58.810844 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:58.810908 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:37747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 11:59:58.811758 31189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.811544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:37747 (local address 127.0.0.1:35078)
0811 11:59:58.811651 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:58.811707 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:37747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 11:59:58.811897 31185 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.143.188:37747: Network error: Client connection negotiation failed: client connection to 127.6.143.188:37747: connect: Connection refused (error 111)
W20260811 11:59:58.827631 31272 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 11:59:58.827939 31272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 11:59:58.828014 31272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 11:59:58.828083 31272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 11:59:58.831066 31265 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.143.188" port: 37747 } attempt: 2
I20260811 11:59:58.831801 31268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.831493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:37747 (local address 127.0.0.1:35088)
0811 11:59:58.831660 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:58.831723 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:37747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 11:59:58.832437 31272 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 11:59:58.832494 31272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 11:59:58.832523 31272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 11:59:58.832551 31272 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 11:59:58.832572 31272 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 11:59:58.832640 31268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.832385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:37747 (local address 127.0.0.1:35092)
0811 11:59:58.832512 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:58.832570 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:37747: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 11:59:58.832747 31265 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.143.188:37747: Network error: Client connection negotiation failed: client connection to 127.6.143.188:37747: connect: Connection refused (error 111)
I20260811 11:59:58.837779 31272 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:39687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:38021,127.6.143.189:36167,127.6.143.188:37747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.188.keytab
--principal=kudu/127.6.143.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.188:37747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.143.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 11:59:58.838717 31272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 11:59:58.839696 31272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 11:59:58.847940 31278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:58.847978 31277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 11:59:58.848035 31280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 11:59:58.848124 31272 server_base.cc:1034] running on GCE node
I20260811 11:59:58.848827 31272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 11:59:58.849843 31272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 11:59:58.851044 31272 hybrid_clock.cc:648] HybridClock initialized: now 1786449598851031 us; error 24 us; skew 500 ppm
Aug 11 11:59:58 dist-test-slave-zkpd krb5kdc[31097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449598, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 11:59:58.853715 31272 init.cc:377] Logged in from keytab as kudu/127.6.143.188@KRBTEST.COM (short username kudu)
I20260811 11:59:58.854549 31272 webserver.cc:460] Webserver started at http://127.6.143.188:46343/ using document root <none> and password file <none>
I20260811 11:59:58.855046 31272 fs_manager.cc:359] Metadata directory not provided
I20260811 11:59:58.855103 31272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 11:59:58.855259 31272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 11:59:58.857206 31272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-2/data/instance:
uuid: "ba413ceb75a74e298f4a32f485eb4e80"
format_stamp: "Formatted at 2026-08-11 11:59:58 on dist-test-slave-zkpd"
server_key: "44cba7a379645ad80fc42ab6f6f8bd82"
server_key_iv: "73d0350673da5578afcee31d9382a3b4"
server_key_version: "encryptionkey@0"
I20260811 11:59:58.857695 31272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-2/wal/instance:
uuid: "ba413ceb75a74e298f4a32f485eb4e80"
format_stamp: "Formatted at 2026-08-11 11:59:58 on dist-test-slave-zkpd"
server_key: "44cba7a379645ad80fc42ab6f6f8bd82"
server_key_iv: "73d0350673da5578afcee31d9382a3b4"
server_key_version: "encryptionkey@0"
I20260811 11:59:58.861228 31272 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 11:59:58.863507 31289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 11:59:58.864780 31272 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 11:59:58.864888 31272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-2/wal
uuid: "ba413ceb75a74e298f4a32f485eb4e80"
format_stamp: "Formatted at 2026-08-11 11:59:58 on dist-test-slave-zkpd"
server_key: "44cba7a379645ad80fc42ab6f6f8bd82"
server_key_iv: "73d0350673da5578afcee31d9382a3b4"
server_key_version: "encryptionkey@0"
I20260811 11:59:58.864975 31272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 11:59:58.876646 31272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 11:59:58.879370 31272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 11:59:58.879552 31272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 11:59:58.886467 31272 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.188:37747
I20260811 11:59:58.886835 31345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.188:37747 every 8 connection(s)
I20260811 11:59:58.887408 31272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-2/data/info.pb
I20260811 11:59:58.889832 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 31272
I20260811 11:59:58.889978 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786449319379472-6718-0/minicluster-data/master-2/wal/instance
I20260811 11:59:58.890316 6718 external_mini_cluster.cc:1428] Setting key 6ee18d89534e70f225ee009cdcd297a8
I20260811 11:59:58.890719 31346 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 11:59:58.895767 31346 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38021 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 11:59:58 dist-test-slave-zkpd krb5kdc[31097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.188@KRBTEST.COM
Aug 11 11:59:58 dist-test-slave-zkpd krb5kdc[31097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449598, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.188@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 11:59:58.905342 31349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.891603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35104 (local address 127.6.143.188:37747)
0811 11:59:58.892060 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:58.892077 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:58.892110 (+ 33us) server_negotiation.cc:407] Connection header received
0811 11:59:58.892759 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:58.892774 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:58.893074 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:58.893338 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:58.894434 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.895374 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.896661 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.897108 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.897225 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:58.900077 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:58.900082 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:58.900091 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:58.900117 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:58.901733 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:58.902195 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:58.902205 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:58.902210 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:58.902315 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:58.902611 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:58.902616 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:58.902617 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:58.903102 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:58.903302 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:58.903939 (+ 637us) server_negotiation.cc:299] Negotiation successful
0811 11:59:58.904258 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":156,"threads_started":1}
I20260811 11:59:58.907496 31350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.897695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:38021 (local address 127.0.0.1:41796)
0811 11:59:58.898071 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:58.898125 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:58.898309 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:58.898668 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:58.898675 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:58.899033 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:59:58.899163 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:58.899169 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.900108 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.900120 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:58.901044 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:58.901058 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.901577 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.901581 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:58.901719 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:58.902246 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:59:58.902259 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:59:58.903738 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:59:58.905770 (+ 2032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:58.905776 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:58.905781 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:58.906074 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:58.906357 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:58.906360 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:58.906361 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:58.906474 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:58.906946 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:59:58.906950 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:59:58.907040 (+ 90us) client_negotiation.cc:769] Sending connection context
0811 11:59:58.907148 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 11:59:58.907293 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":125,"threads_started":1}
I20260811 11:59:58.907572 31188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.897784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41796 (local address 127.6.143.190:38021)
0811 11:59:58.897954 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:58.897958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:58.898239 (+ 281us) server_negotiation.cc:407] Connection header received
0811 11:59:58.898433 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:58.898453 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:58.898518 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:58.898614 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:58.899307 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.899977 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.901190 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.901449 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.901574 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:58.903916 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:58.903921 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:58.903924 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:58.903954 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:58.905642 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:58.906176 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:58.906179 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:58.906180 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:58.906262 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:58.906584 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:58.906587 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:58.906589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:58.906833 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:58.906949 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:58.907247 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 11:59:58.907403 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 11:59:58.908787 31346 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.189" port: 36167 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:59:58.911060 31185 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.143.188" port: 37747 } attempt: 3
Aug 11 11:59:58 dist-test-slave-zkpd krb5kdc[31097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
Aug 11 11:59:58 dist-test-slave-zkpd krb5kdc[31097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449598, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.188@KRBTEST.COM for kudu/127.6.143.189@KRBTEST.COM
Aug 11 11:59:58 dist-test-slave-zkpd krb5kdc[31097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449598, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for kudu/127.6.143.188@KRBTEST.COM
I20260811 11:59:58.917845 31188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.908017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41812 (local address 127.6.143.190:38021)
0811 11:59:58.908131 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:58.908135 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:58.908150 (+ 15us) server_negotiation.cc:407] Connection header received
0811 11:59:58.908276 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:58.908280 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:58.908339 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:58.908429 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:58.909011 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.909583 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.910559 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.910858 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.910975 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:58.913936 (+ 2961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:58.913940 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:58.913942 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:58.913967 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:58.915937 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:58.916451 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:58.916457 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:58.916459 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:58.916553 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:58.916889 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:58.916893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:58.916895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:58.917120 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:58.917386 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:58.917531 (+ 145us) server_negotiation.cc:299] Negotiation successful
0811 11:59:58.917656 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 11:59:58.918601 31350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.909276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:36167 (local address 127.0.0.1:50084)
0811 11:59:58.909403 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:58.909421 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:58.909496 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:58.909871 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:58.909875 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:58.910047 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:59:58.910155 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:58.910161 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.911003 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.911007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:58.912173 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:58.912193 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.912817 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.912821 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:58.912923 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:58.913349 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:59:58.913370 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:59:58.914689 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:59:58.916950 (+ 2261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:58.916956 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:58.916960 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:58.917200 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:58.917547 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:58.917551 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:58.917554 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:58.917644 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:58.918113 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:59:58.918116 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:59:58.918221 (+ 105us) client_negotiation.cc:769] Sending connection context
0811 11:59:58.918327 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 11:59:58.918458 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 11:59:58.918635 31269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.909500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50084 (local address 127.6.143.189:36167)
0811 11:59:58.909655 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:58.909661 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:58.909676 (+ 15us) server_negotiation.cc:407] Connection header received
0811 11:59:58.909726 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:58.909728 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:58.909764 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:58.909821 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:58.910279 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.910899 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.912305 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.912698 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.912814 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:58.914854 (+ 2040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:58.914860 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:58.914862 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:58.914891 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:58.916627 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:58.917324 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:58.917328 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:58.917331 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:58.917427 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:58.917727 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:58.917731 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:58.917733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:58.917965 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:58.918066 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:58.918385 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 11:59:58.918495 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 11:59:58.919548 31346 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.188" port: 37747 } has no permanent_uuid. Determining permanent_uuid...
I20260811 11:59:58.921293 31189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.911432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:37747 (local address 127.0.0.1:35120)
0811 11:59:58.911762 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:58.911777 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:58.911862 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:58.912183 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:58.912186 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:58.912378 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:59:58.912455 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:58.912461 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.913574 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.913578 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:58.914388 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:58.914402 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.915036 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.915040 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:58.915128 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:58.915680 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:59:58.915696 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:59:58.917248 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:59:58.919536 (+ 2288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:58.919540 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:58.919542 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:58.919812 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:58.920187 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:58.920192 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:58.920209 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:58.920305 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:58.920817 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:59:58.920822 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:59:58.920924 (+ 102us) client_negotiation.cc:769] Sending connection context
0811 11:59:58.921032 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 11:59:58.921151 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206}
I20260811 11:59:58.921303 31349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.911586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35120 (local address 127.6.143.188:37747)
0811 11:59:58.911730 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:58.911754 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:58.911847 (+ 93us) server_negotiation.cc:407] Connection header received
0811 11:59:58.911982 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:58.911987 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:58.912058 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:58.912151 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:58.912578 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.913415 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.914556 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.914887 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.915363 (+ 476us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:58.917388 (+ 2025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:58.917392 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:58.917394 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:58.917430 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:58.919421 (+ 1991us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:58.919948 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:58.919952 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:58.919953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:58.920049 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:58.920435 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:58.920437 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:58.920439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:58.920698 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:58.920849 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:58.921083 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 11:59:58.921181 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":55}
I20260811 11:59:58.923992 31265 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.143.188" port: 37747 } attempt: 3
Aug 11 11:59:58 dist-test-slave-zkpd krb5kdc[31097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449598, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.188@KRBTEST.COM for kudu/127.6.143.188@KRBTEST.COM
I20260811 11:59:58.926393 31185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bf3f8e00a484315b3ed43089ac18ee3: Bootstrap starting.
Aug 11 11:59:58 dist-test-slave-zkpd krb5kdc[31097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449598, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.189@KRBTEST.COM
Aug 11 11:59:58 dist-test-slave-zkpd krb5kdc[31097](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449598, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.189@KRBTEST.COM for kudu/127.6.143.188@KRBTEST.COM
I20260811 11:59:58.929899 31185 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5bf3f8e00a484315b3ed43089ac18ee3: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:58.930334 31350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.919994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:37747 (local address 127.0.0.1:35122)
0811 11:59:58.920235 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:58.920249 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:58.920316 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:58.920632 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:58.920638 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:58.920858 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:59:58.920952 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:58.920959 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.922582 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.922585 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:58.923201 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:58.923210 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.923816 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.923819 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:58.923884 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:58.924845 (+ 961us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:59:58.924860 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:59:58.926226 (+ 1366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:59:58.928666 (+ 2440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:58.928669 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:58.928674 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:58.928901 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:58.929188 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:58.929190 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:58.929192 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:58.929282 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:58.929728 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:59:58.929731 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:59:58.929892 (+ 161us) client_negotiation.cc:769] Sending connection context
0811 11:59:58.930048 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 11:59:58.930171 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145}
I20260811 11:59:58.931038 31351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.920050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35122 (local address 127.6.143.188:37747)
0811 11:59:58.920391 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:58.920401 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:58.920413 (+ 12us) server_negotiation.cc:407] Connection header received
0811 11:59:58.920439 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:58.920441 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:58.920496 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:58.920583 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:58.921080 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.922472 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.923305 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.923636 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.923731 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:58.926378 (+ 2647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:58.926381 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:58.926383 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:58.926415 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:58.928556 (+ 2141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:58.929002 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:58.929004 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:58.929006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:58.929086 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:58.929376 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:58.929378 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:58.929380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:58.929607 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:58.929696 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:58.930714 (+ 1018us) server_negotiation.cc:299] Negotiation successful
0811 11:59:58.930833 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":269,"thread_start_us":132,"threads_started":1}
I20260811 11:59:58.931291 31185 log.cc:824] T 00000000000000000000000000000000 P 5bf3f8e00a484315b3ed43089ac18ee3: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:58.932319 31269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.919725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50086 (local address 127.6.143.189:36167)
0811 11:59:58.919833 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:58.919839 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:58.919859 (+ 20us) server_negotiation.cc:407] Connection header received
0811 11:59:58.920076 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:58.920080 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:58.920143 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:58.920214 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:58.921192 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.922101 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.923663 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.924056 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.924157 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:58.927246 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:58.927250 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:58.927253 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:58.927279 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:58.929366 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:58.929945 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:58.929952 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:58.929955 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:58.930065 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:58.930799 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:58.930807 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:58.930809 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:58.931053 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:58.931536 (+ 483us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:58.932013 (+ 477us) server_negotiation.cc:299] Negotiation successful
0811 11:59:58.932129 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 11:59:58.933933 31268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.924370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:37747 (local address 127.0.0.1:35124)
0811 11:59:58.924632 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0811 11:59:58.924646 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 11:59:58.924717 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 11:59:58.924982 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 11:59:58.924987 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 11:59:58.925146 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0811 11:59:58.925249 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:58.925260 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.926385 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.926389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:58.927278 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 11:59:58.927294 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.928043 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.928047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 11:59:58.928149 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:58.928543 (+ 394us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 11:59:58.928558 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 11:59:58.930150 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 11:59:58.932222 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:58.932225 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:58.932227 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:58.932484 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:58.932827 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 11:59:58.932831 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 11:59:58.932833 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 11:59:58.932937 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 11:59:58.933380 (+ 443us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 11:59:58.933385 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 11:59:58.933505 (+ 120us) client_negotiation.cc:769] Sending connection context
0811 11:59:58.933621 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 11:59:58.933782 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191}
I20260811 11:59:58.934502 31349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.924613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35124 (local address 127.6.143.188:37747)
0811 11:59:58.924706 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:58.924710 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:58.924723 (+ 13us) server_negotiation.cc:407] Connection header received
0811 11:59:58.924793 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:58.924799 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:58.924840 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:58.925375 (+ 535us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:58.925473 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.926260 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.927475 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.927862 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.928344 (+ 482us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:58.930286 (+ 1942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:58.930289 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:58.930292 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:58.930312 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:58.932102 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:58.932612 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:58.932616 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:58.932619 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:58.932701 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:58.933057 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:58.933060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:58.933062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:58.933260 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:58.933361 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:58.934252 (+ 891us) server_negotiation.cc:299] Negotiation successful
0811 11:59:58.934367 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 11:59:58.934553 31346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba413ceb75a74e298f4a32f485eb4e80: Bootstrap starting.
I20260811 11:59:58.936092 6718 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 11:59:58.937520 31346 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba413ceb75a74e298f4a32f485eb4e80: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:58.937708 31265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57ecbe71f3b04dd592049175623572d3: Bootstrap starting.
I20260811 11:59:58.937314 31157 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "5bf3f8e00a484315b3ed43089ac18ee3"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41812
I20260811 11:59:58.938493 31346 log.cc:824] T 00000000000000000000000000000000 P ba413ceb75a74e298f4a32f485eb4e80: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:58.940665 31265 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57ecbe71f3b04dd592049175623572d3: Neither blocks nor log segments found. Creating new log.
I20260811 11:59:58.940934 31185 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bf3f8e00a484315b3ed43089ac18ee3: No bootstrap required, opened a new log
I20260811 11:59:58.940961 31346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba413ceb75a74e298f4a32f485eb4e80: No bootstrap required, opened a new log
I20260811 11:59:58.941697 31265 log.cc:824] T 00000000000000000000000000000000 P 57ecbe71f3b04dd592049175623572d3: Log is configured to *not* fsync() on all Append() calls
I20260811 11:59:58.943866 31265 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57ecbe71f3b04dd592049175623572d3: No bootstrap required, opened a new log
I20260811 11:59:58.944072 31185 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5bf3f8e00a484315b3ed43089ac18ee3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf3f8e00a484315b3ed43089ac18ee3" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38021 } } peers { permanent_uuid: "57ecbe71f3b04dd592049175623572d3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 36167 } } peers { permanent_uuid: "ba413ceb75a74e298f4a32f485eb4e80" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 37747 } }
I20260811 11:59:58.944285 31185 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5bf3f8e00a484315b3ed43089ac18ee3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bf3f8e00a484315b3ed43089ac18ee3, State: Initialized, Role: FOLLOWER
I20260811 11:59:58.944608 31185 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5bf3f8e00a484315b3ed43089ac18ee3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf3f8e00a484315b3ed43089ac18ee3" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38021 } } peers { permanent_uuid: "57ecbe71f3b04dd592049175623572d3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 36167 } } peers { permanent_uuid: "ba413ceb75a74e298f4a32f485eb4e80" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 37747 } }
I20260811 11:59:58.945330 31346 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba413ceb75a74e298f4a32f485eb4e80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf3f8e00a484315b3ed43089ac18ee3" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38021 } } peers { permanent_uuid: "57ecbe71f3b04dd592049175623572d3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 36167 } } peers { permanent_uuid: "ba413ceb75a74e298f4a32f485eb4e80" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 37747 } }
I20260811 11:59:58.945596 31346 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba413ceb75a74e298f4a32f485eb4e80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba413ceb75a74e298f4a32f485eb4e80, State: Initialized, Role: FOLLOWER
I20260811 11:59:58.945892 31185 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5bf3f8e00a484315b3ed43089ac18ee3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:59:58.945950 31157 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5bf3f8e00a484315b3ed43089ac18ee3 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 11:59:58.945931 31346 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba413ceb75a74e298f4a32f485eb4e80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf3f8e00a484315b3ed43089ac18ee3" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38021 } } peers { permanent_uuid: "57ecbe71f3b04dd592049175623572d3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 36167 } } peers { permanent_uuid: "ba413ceb75a74e298f4a32f485eb4e80" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 37747 } }
I20260811 11:59:58.946041 31157 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5bf3f8e00a484315b3ed43089ac18ee3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:58.947214 31346 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba413ceb75a74e298f4a32f485eb4e80 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:59:58.947363 31157 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5bf3f8e00a484315b3ed43089ac18ee3 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf3f8e00a484315b3ed43089ac18ee3" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38021 } } peers { permanent_uuid: "57ecbe71f3b04dd592049175623572d3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 36167 } } peers { permanent_uuid: "ba413ceb75a74e298f4a32f485eb4e80" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 37747 } }
I20260811 11:59:58.948449 31356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bf3f8e00a484315b3ed43089ac18ee3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf3f8e00a484315b3ed43089ac18ee3" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38021 } } peers { permanent_uuid: "57ecbe71f3b04dd592049175623572d3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 36167 } } peers { permanent_uuid: "ba413ceb75a74e298f4a32f485eb4e80" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 37747 } } }
I20260811 11:59:58.948570 31356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bf3f8e00a484315b3ed43089ac18ee3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:59:58.949124 31157 leader_election.cc:288] T 00000000000000000000000000000000 P 5bf3f8e00a484315b3ed43089ac18ee3 [CANDIDATE]: Term 1 election: Requested vote from peers 57ecbe71f3b04dd592049175623572d3 (127.6.143.189:36167), ba413ceb75a74e298f4a32f485eb4e80 (127.6.143.188:37747)
I20260811 11:59:58.948200 31265 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57ecbe71f3b04dd592049175623572d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf3f8e00a484315b3ed43089ac18ee3" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38021 } } peers { permanent_uuid: "57ecbe71f3b04dd592049175623572d3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 36167 } } peers { permanent_uuid: "ba413ceb75a74e298f4a32f485eb4e80" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 37747 } }
I20260811 11:59:58.950317 31265 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57ecbe71f3b04dd592049175623572d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57ecbe71f3b04dd592049175623572d3, State: Initialized, Role: FOLLOWER
I20260811 11:59:58.950651 31265 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57ecbe71f3b04dd592049175623572d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf3f8e00a484315b3ed43089ac18ee3" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38021 } } peers { permanent_uuid: "57ecbe71f3b04dd592049175623572d3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 36167 } } peers { permanent_uuid: "ba413ceb75a74e298f4a32f485eb4e80" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 37747 } }
I20260811 11:59:58.951481 31318 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5bf3f8e00a484315b3ed43089ac18ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ba413ceb75a74e298f4a32f485eb4e80"
I20260811 11:59:58.951682 31318 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba413ceb75a74e298f4a32f485eb4e80 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:58.952028 31265 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57ecbe71f3b04dd592049175623572d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 11:59:58.953011 31318 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ba413ceb75a74e298f4a32f485eb4e80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bf3f8e00a484315b3ed43089ac18ee3 in term 1.
I20260811 11:59:58.953152 31388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57ecbe71f3b04dd592049175623572d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf3f8e00a484315b3ed43089ac18ee3" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38021 } } peers { permanent_uuid: "57ecbe71f3b04dd592049175623572d3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 36167 } } peers { permanent_uuid: "ba413ceb75a74e298f4a32f485eb4e80" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 37747 } } }
I20260811 11:59:58.953302 31388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57ecbe71f3b04dd592049175623572d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:59:58.953550 31357 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba413ceb75a74e298f4a32f485eb4e80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf3f8e00a484315b3ed43089ac18ee3" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38021 } } peers { permanent_uuid: "57ecbe71f3b04dd592049175623572d3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 36167 } } peers { permanent_uuid: "ba413ceb75a74e298f4a32f485eb4e80" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 37747 } } }
I20260811 11:59:58.953711 31357 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba413ceb75a74e298f4a32f485eb4e80 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 11:59:58.954068 31370 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5bf3f8e00a484315b3ed43089ac18ee3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 11:59:58.954130 31370 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 11:59:58.954912 31129 leader_election.cc:302] T 00000000000000000000000000000000 P 5bf3f8e00a484315b3ed43089ac18ee3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5bf3f8e00a484315b3ed43089ac18ee3, ba413ceb75a74e298f4a32f485eb4e80; no voters:
I20260811 11:59:58.955291 31356 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5bf3f8e00a484315b3ed43089ac18ee3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 11:59:58.955582 31356 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5bf3f8e00a484315b3ed43089ac18ee3 [term 1 LEADER]: Becoming Leader. State: Replica: 5bf3f8e00a484315b3ed43089ac18ee3, State: Running, Role: LEADER
I20260811 11:59:58.955979 31356 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5bf3f8e00a484315b3ed43089ac18ee3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf3f8e00a484315b3ed43089ac18ee3" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38021 } } peers { permanent_uuid: "57ecbe71f3b04dd592049175623572d3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 36167 } } peers { permanent_uuid: "ba413ceb75a74e298f4a32f485eb4e80" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 37747 } }
I20260811 11:59:58.957852 31356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bf3f8e00a484315b3ed43089ac18ee3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bf3f8e00a484315b3ed43089ac18ee3. Latest consensus state: current_term: 1 leader_uuid: "5bf3f8e00a484315b3ed43089ac18ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf3f8e00a484315b3ed43089ac18ee3" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38021 } } peers { permanent_uuid: "57ecbe71f3b04dd592049175623572d3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 36167 } } peers { permanent_uuid: "ba413ceb75a74e298f4a32f485eb4e80" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 37747 } } }
I20260811 11:59:58.958027 31356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bf3f8e00a484315b3ed43089ac18ee3 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:58.954818 31272 master_runner.cc:186] Error getting master registration for 127.6.143.189:36167: OK, instance_id { permanent_uuid: "57ecbe71f3b04dd592049175623572d3" instance_seqno: 1786449598671964 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 11:59:58.954818 31110 master_runner.cc:186] Error getting master registration for 127.6.143.189:36167: OK, instance_id { permanent_uuid: "57ecbe71f3b04dd592049175623572d3" instance_seqno: 1786449598671964 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 11:59:58.958420 31272 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 11:59:58.954372 31237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5bf3f8e00a484315b3ed43089ac18ee3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "57ecbe71f3b04dd592049175623572d3"
I20260811 11:59:58.959642 31237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57ecbe71f3b04dd592049175623572d3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 11:59:58.961233 31237 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 57ecbe71f3b04dd592049175623572d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5bf3f8e00a484315b3ed43089ac18ee3 in term 1.
W20260811 11:59:58.962960 31381 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ba413ceb75a74e298f4a32f485eb4e80: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 11:59:58.963081 31381 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 11:59:58.963423 31404 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 11:59:58.964435 31404 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 11:59:58.966809 31110 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 11:59:58.969543 31269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.958545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50096 (local address 127.6.143.189:36167)
0811 11:59:58.958660 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:58.958667 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:58.959118 (+ 451us) server_negotiation.cc:407] Connection header received
0811 11:59:58.959336 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:58.959341 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:58.959393 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:58.959450 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:58.960616 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.961414 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.962631 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.962979 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.963059 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:58.965102 (+ 2043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:58.965107 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:58.965110 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:58.965138 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:58.967086 (+ 1948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:58.967684 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:58.967688 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:58.967690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:58.967808 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:58.968238 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:58.968242 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:58.968244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:58.968508 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:58.968629 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:58.969154 (+ 525us) server_negotiation.cc:299] Negotiation successful
0811 11:59:58.969319 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 11:59:58.973533 31188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.953313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41816 (local address 127.6.143.190:38021)
0811 11:59:58.953463 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:58.953470 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:58.954440 (+ 970us) server_negotiation.cc:407] Connection header received
0811 11:59:58.954640 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:58.954643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:58.954691 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:58.954743 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:58.955820 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.956588 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.957754 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.958089 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.958183 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:58.968081 (+ 9898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:58.968085 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:58.968088 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:58.968118 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:58.971463 (+ 3345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:58.972031 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:58.972035 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:58.972037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:58.972132 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:58.972509 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:58.972513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:58.972515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:58.972744 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:58.972854 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:58.973256 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 11:59:58.973386 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260811 11:59:58.974737 31401 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 57ecbe71f3b04dd592049175623572d3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 11:59:58.974826 31401 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 11:59:58.983992 31318 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ba413ceb75a74e298f4a32f485eb4e80 [term 1 FOLLOWER]: Refusing update from remote peer 5bf3f8e00a484315b3ed43089ac18ee3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:59:58.984279 31237 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 57ecbe71f3b04dd592049175623572d3 [term 1 FOLLOWER]: Refusing update from remote peer 5bf3f8e00a484315b3ed43089ac18ee3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 11:59:58.984809 31356 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5bf3f8e00a484315b3ed43089ac18ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba413ceb75a74e298f4a32f485eb4e80" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 37747 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:59:58.985205 31356 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5bf3f8e00a484315b3ed43089ac18ee3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57ecbe71f3b04dd592049175623572d3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 36167 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 11:59:58.989030 31388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57ecbe71f3b04dd592049175623572d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bf3f8e00a484315b3ed43089ac18ee3. Latest consensus state: current_term: 1 leader_uuid: "5bf3f8e00a484315b3ed43089ac18ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf3f8e00a484315b3ed43089ac18ee3" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38021 } } peers { permanent_uuid: "57ecbe71f3b04dd592049175623572d3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 36167 } } peers { permanent_uuid: "ba413ceb75a74e298f4a32f485eb4e80" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 37747 } } }
I20260811 11:59:58.989141 31388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57ecbe71f3b04dd592049175623572d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:59:58.993779 31349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 11:59:58.959110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35138 (local address 127.6.143.188:37747)
0811 11:59:58.959216 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:58.959220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:58.967723 (+ 8503us) server_negotiation.cc:407] Connection header received
0811 11:59:58.967931 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:58.967934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:58.967982 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:58.968035 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:58.969088 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.969855 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.971325 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 11:59:58.971701 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 11:59:58.971780 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 11:59:58.981597 (+ 9817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 11:59:58.981601 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 11:59:58.981604 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 11:59:58.981631 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 11:59:58.983280 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:58.992076 (+ 8796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:58.992079 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:58.992081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:58.992172 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 11:59:58.992508 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 11:59:58.992512 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 11:59:58.992514 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 11:59:58.992732 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 11:59:58.992851 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 11:59:58.993490 (+ 639us) server_negotiation.cc:299] Negotiation successful
0811 11:59:58.993605 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 11:59:58.994846 31357 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba413ceb75a74e298f4a32f485eb4e80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bf3f8e00a484315b3ed43089ac18ee3. Latest consensus state: current_term: 1 leader_uuid: "5bf3f8e00a484315b3ed43089ac18ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf3f8e00a484315b3ed43089ac18ee3" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38021 } } peers { permanent_uuid: "57ecbe71f3b04dd592049175623572d3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 36167 } } peers { permanent_uuid: "ba413ceb75a74e298f4a32f485eb4e80" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 37747 } } }
I20260811 11:59:58.994958 31357 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba413ceb75a74e298f4a32f485eb4e80 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:59:59.001989 31402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bf3f8e00a484315b3ed43089ac18ee3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5bf3f8e00a484315b3ed43089ac18ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf3f8e00a484315b3ed43089ac18ee3" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38021 } } peers { permanent_uuid: "57ecbe71f3b04dd592049175623572d3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 36167 } } peers { permanent_uuid: "ba413ceb75a74e298f4a32f485eb4e80" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 37747 } } }
I20260811 11:59:59.002131 31402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bf3f8e00a484315b3ed43089ac18ee3 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:59.002262 31408 mvcc.cc:205] Tried to move back new op lower bound from 7317297557429538816 to 7317297557327613952. Current Snapshot: MvccSnapshot[applied={T|T < 7317297557429538816}]
I20260811 11:59:59.003685 31388 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57ecbe71f3b04dd592049175623572d3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5bf3f8e00a484315b3ed43089ac18ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf3f8e00a484315b3ed43089ac18ee3" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38021 } } peers { permanent_uuid: "57ecbe71f3b04dd592049175623572d3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 36167 } } peers { permanent_uuid: "ba413ceb75a74e298f4a32f485eb4e80" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 37747 } } }
I20260811 11:59:59.003793 31388 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57ecbe71f3b04dd592049175623572d3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:59:59.004494 31410 mvcc.cc:205] Tried to move back new op lower bound from 7317297557429538816 to 7317297557327613952. Current Snapshot: MvccSnapshot[applied={T|T < 7317297557429538816}]
I20260811 11:59:59.005200 31356 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bf3f8e00a484315b3ed43089ac18ee3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5bf3f8e00a484315b3ed43089ac18ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf3f8e00a484315b3ed43089ac18ee3" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38021 } } peers { permanent_uuid: "57ecbe71f3b04dd592049175623572d3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 36167 } } peers { permanent_uuid: "ba413ceb75a74e298f4a32f485eb4e80" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 37747 } } }
I20260811 11:59:59.005306 31356 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bf3f8e00a484315b3ed43089ac18ee3 [sys.catalog]: This master's current role is: LEADER
I20260811 11:59:59.006371 31357 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba413ceb75a74e298f4a32f485eb4e80 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5bf3f8e00a484315b3ed43089ac18ee3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5bf3f8e00a484315b3ed43089ac18ee3" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38021 } } peers { permanent_uuid: "57ecbe71f3b04dd592049175623572d3" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 36167 } } peers { permanent_uuid: "ba413ceb75a74e298f4a32f485eb4e80" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 37747 } } }
I20260811 11:59:59.006489 31357 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba413ceb75a74e298f4a32f485eb4e80 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 11:59:59.007011 31412 mvcc.cc:205] Tried to move back new op lower bound from 7317297557429538816 to 7317297557327613952. Current Snapshot: MvccSnapshot[applied={T|T < 7317297557429538816}]
I20260811 11:59:59.012035 31404 catalog_manager.cc:1347] Generated new cluster ID: 35690d6deb344810b2905c7afe0cd717
I20260811 11:59:59.012097 31404 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 11:59:59.024863 31404 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 11:59:59.026306 31404 catalog_manager.cc:1504] Loading token signing keys...
I20260811 11:59:59.047298 31404 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5bf3f8e00a484315b3ed43089ac18ee3: Generated new TSK 0
I20260811 11:59:59.048244 31404 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 11:59:59.079222 31269 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:59:59.076200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50112 (local address 127.6.143.189:36167)
0811 11:59:59.076332 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:59.076337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:59.078159 (+ 1822us) server_negotiation.cc:407] Connection header received
0811 11:59:59.078391 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:59.078394 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:59.078453 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:59.078505 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:59.079093 (+ 588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50112: BlockingRecv error: recv got EOF from 127.0.0.1:50112 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 11:59:59.081122 31349 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:59:59.076932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35140 (local address 127.6.143.188:37747)
0811 11:59:59.077048 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:59.077052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:59.079752 (+ 2700us) server_negotiation.cc:407] Connection header received
0811 11:59:59.080285 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:59.080288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:59.080333 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:59.080379 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:59.081005 (+ 626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35140: BlockingRecv error: recv got EOF from 127.0.0.1:35140 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 11:59:59.082211 31188 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:59:59.077584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41826 (local address 127.6.143.190:38021)
0811 11:59:59.077691 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:59.077695 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:59.080102 (+ 2407us) server_negotiation.cc:407] Connection header received
0811 11:59:59.081483 (+ 1381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:59.081487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:59.081545 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:59.081623 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:59.082103 (+ 480us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41826: BlockingRecv error: recv got EOF from 127.0.0.1:41826 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260811 11:59:59.086771 31269 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:59:59.085262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50118 (local address 127.6.143.189:36167)
0811 11:59:59.085359 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:59.085363 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:59.085827 (+ 464us) server_negotiation.cc:407] Connection header received
0811 11:59:59.086055 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:59.086059 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:59.086148 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:59.086198 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:59.086682 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50118: BlockingRecv error: recv got EOF from 127.0.0.1:50118 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260811 11:59:59.091495 31269 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:59:59.090003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50132 (local address 127.6.143.189:36167)
0811 11:59:59.090097 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:59.090100 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:59.090622 (+ 522us) server_negotiation.cc:407] Connection header received
0811 11:59:59.090809 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:59.090814 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:59.090859 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:59.090909 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:59.091390 (+ 481us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50132: BlockingRecv error: recv got EOF from 127.0.0.1:50132 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260811 11:59:59.096987 31269 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:59:59.095496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50142 (local address 127.6.143.189:36167)
0811 11:59:59.095589 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:59.095593 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:59.096123 (+ 530us) server_negotiation.cc:407] Connection header received
0811 11:59:59.096314 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:59.096317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:59.096363 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:59.096430 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:59.096895 (+ 465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50142: BlockingRecv error: recv got EOF from 127.0.0.1:50142 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260811 11:59:59.104661 31269 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:59:59.103037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50150 (local address 127.6.143.189:36167)
0811 11:59:59.103165 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:59.103169 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:59.103687 (+ 518us) server_negotiation.cc:407] Connection header received
0811 11:59:59.103871 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:59.103874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:59.103949 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:59.104006 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:59.104552 (+ 546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50150: BlockingRecv error: recv got EOF from 127.0.0.1:50150 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260811 11:59:59.116430 31269 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:59:59.114753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50154 (local address 127.6.143.189:36167)
0811 11:59:59.114850 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:59.114854 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:59.115425 (+ 571us) server_negotiation.cc:407] Connection header received
0811 11:59:59.115631 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:59.115635 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:59.115683 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:59.115735 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:59.116324 (+ 589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50154: BlockingRecv error: recv got EOF from 127.0.0.1:50154 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260811 11:59:59.136759 31269 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:59:59.134764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50160 (local address 127.6.143.189:36167)
0811 11:59:59.134882 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:59.134886 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:59.135481 (+ 595us) server_negotiation.cc:407] Connection header received
0811 11:59:59.135712 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:59.135716 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:59.135766 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:59.135817 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:59.136652 (+ 835us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50160: BlockingRecv error: recv got EOF from 127.0.0.1:50160 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 11:59:59.172755 31269 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:59:59.171251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50168 (local address 127.6.143.189:36167)
0811 11:59:59.171397 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:59.171403 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:59.171714 (+ 311us) server_negotiation.cc:407] Connection header received
0811 11:59:59.171932 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:59.171936 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:59.171999 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:59.172093 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:59.172620 (+ 527us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50168: BlockingRecv error: recv got EOF from 127.0.0.1:50168 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 11:59:59.240356 31269 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:59:59.239001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50170 (local address 127.6.143.189:36167)
0811 11:59:59.239170 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:59.239175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:59.239336 (+ 161us) server_negotiation.cc:407] Connection header received
0811 11:59:59.239525 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:59.239529 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:59.239578 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:59.239673 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:59.240229 (+ 556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50170: BlockingRecv error: recv got EOF from 127.0.0.1:50170 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 11:59:59.372030 31269 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:59:59.370578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50178 (local address 127.6.143.189:36167)
0811 11:59:59.370721 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:59.370750 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:59.371008 (+ 258us) server_negotiation.cc:407] Connection header received
0811 11:59:59.371197 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:59.371200 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:59.371249 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:59.371340 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:59.371841 (+ 501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50178: BlockingRecv error: recv got EOF from 127.0.0.1:50178 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 11:59:59.631444 31269 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 11:59:59.630354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50190 (local address 127.6.143.189:36167)
0811 11:59:59.630501 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 11:59:59.630506 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 11:59:59.630581 (+ 75us) server_negotiation.cc:407] Connection header received
0811 11:59:59.630692 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 11:59:59.630694 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 11:59:59.630741 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 11:59:59.630823 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 11:59:59.631300 (+ 477us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50190: BlockingRecv error: recv got EOF from 127.0.0.1:50190 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 11:59:59.964505 31381 catalog_manager.cc:1259] Loaded cluster ID: 35690d6deb344810b2905c7afe0cd717
I20260811 11:59:59.964586 31381 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ba413ceb75a74e298f4a32f485eb4e80: loading cluster ID for follower catalog manager: success
I20260811 11:59:59.966305 31381 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ba413ceb75a74e298f4a32f485eb4e80: acquiring CA information for follower catalog manager: success
I20260811 11:59:59.967494 31381 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ba413ceb75a74e298f4a32f485eb4e80: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 11:59:59.976001 31401 catalog_manager.cc:1259] Loaded cluster ID: 35690d6deb344810b2905c7afe0cd717
I20260811 11:59:59.976096 31401 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 57ecbe71f3b04dd592049175623572d3: loading cluster ID for follower catalog manager: success
I20260811 11:59:59.977671 31401 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 57ecbe71f3b04dd592049175623572d3: acquiring CA information for follower catalog manager: success
I20260811 11:59:59.978776 31401 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 57ecbe71f3b04dd592049175623572d3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 12:00:00.149880 31491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:00.145784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50204 (local address 127.6.143.189:36167)
0811 12:00:00.146019 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:00.146023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:00.146038 (+ 15us) server_negotiation.cc:407] Connection header received
0811 12:00:00.146095 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:00.146097 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:00.146232 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:00.146345 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:00:00.146588 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:00.147294 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:00.148377 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:00.148621 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:00.148706 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:00.149014 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:00:00.149542 (+ 528us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:00:00.149641 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 12:00:00.149701 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 12:00:00.149752 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":80,"threads_started":1}
I20260811 12:00:00.156798 31497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:00.153191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35146 (local address 127.6.143.188:37747)
0811 12:00:00.153417 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:00.153420 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:00.153434 (+ 14us) server_negotiation.cc:407] Connection header received
0811 12:00:00.153558 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:00.153560 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:00.153692 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:00.153862 (+ 170us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:00:00.154162 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:00.154674 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:00.155431 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:00.155647 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:00.155726 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:00.156033 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:00:00.156458 (+ 425us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:00:00.156554 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 12:00:00.156628 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 12:00:00.156678 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":89,"threads_started":1}
I20260811 12:00:00.163712 31491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:00.159753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50214 (local address 127.6.143.189:36167)
0811 12:00:00.159941 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:00.159946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:00.160172 (+ 226us) server_negotiation.cc:407] Connection header received
0811 12:00:00.160312 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:00.160315 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:00.160367 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:00.160443 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:00:00.160740 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:00.161486 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:00.162443 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:00.162756 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:00.162869 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:00.163189 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:00:00.163280 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:00:00.163332 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0811 12:00:00.163503 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 12:00:00.163557 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 12:00:00.164420 31497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:00.160142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35156 (local address 127.6.143.188:37747)
0811 12:00:00.160265 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:00.160271 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:00.160765 (+ 494us) server_negotiation.cc:407] Connection header received
0811 12:00:00.160946 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:00.160949 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:00.161001 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:00.161074 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:00:00.161322 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:00.162048 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:00.163156 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:00.163450 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:00.163525 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:00.163950 (+ 425us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:00:00.164028 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:00:00.164107 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 12:00:00.164247 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 12:00:00.164299 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 12:00:00.164887 31506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:00.159673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41842 (local address 127.6.143.190:38021)
0811 12:00:00.160210 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:00.160214 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:00.160228 (+ 14us) server_negotiation.cc:407] Connection header received
0811 12:00:00.160328 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:00.160331 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:00.160494 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:00.160610 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:00:00.160880 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:00.161589 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:00.162649 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:00.162999 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:00.163122 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:00.163423 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:00:00.164162 (+ 739us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:00:00.164249 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 12:00:00.164726 (+ 477us) server_negotiation.cc:299] Negotiation successful
0811 12:00:00.164780 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":73,"threads_started":1}
I20260811 12:00:00.167883 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 31110
I20260811 12:00:00.178386 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 31193
I20260811 12:00:00.185747 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 31272
2026-08-11T12:00:00Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3135 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 12:00:00 dist-test-slave-zkpd krb5kdc[31511](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 12:00:00 dist-test-slave-zkpd krb5kdc[31511](info): set up 2 sockets
Aug 11 12:00:00 dist-test-slave-zkpd krb5kdc[31511](info): commencing operation
krb5kdc: starting...
W20260811 12:00:01.279444 6718 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/krb5kdc/test-user.keytab.
Aug 11 12:00:01 dist-test-slave-zkpd krb5kdc[31511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T12:00:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T12:00:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.143.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
Entry for principal kudu/127.6.143.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab.
I20260811 12:00:01.415431 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:38553
--webserver_interface=127.6.143.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:41533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:38553,127.6.143.189:39709,127.6.143.188:41291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 12:00:01.539512 31524 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 12:00:01.539897 31524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:00:01.540037 31524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:00:01.540118 31524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:00:01.544252 31524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:00:01.544333 31524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:00:01.544370 31524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:00:01.544409 31524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:00:01.544440 31524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:00:01.549957 31524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:41533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:38553,127.6.143.189:39709,127.6.143.188:41291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.190.keytab
--principal=kudu/127.6.143.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.190:38553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.143.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 12:00:01.551122 31524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:00:01.552230 31524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:00:01.561121 31533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:00:01.561110 31531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:00:01.561317 31524 server_base.cc:1034] running on GCE node
W20260811 12:00:01.561187 31535 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:00:01.562114 31524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:00:01.563431 31524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:00:01.564641 31524 hybrid_clock.cc:648] HybridClock initialized: now 1786449601564627 us; error 26 us; skew 500 ppm
Aug 11 12:00:01 dist-test-slave-zkpd krb5kdc[31511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449601, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:00:01.567494 31524 init.cc:377] Logged in from keytab as kudu/127.6.143.190@KRBTEST.COM (short username kudu)
I20260811 12:00:01.568459 31524 webserver.cc:460] Webserver started at http://127.6.143.190:36343/ using document root <none> and password file <none>
I20260811 12:00:01.569010 31524 fs_manager.cc:359] Metadata directory not provided
I20260811 12:00:01.569072 31524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:00:01.569252 31524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:00:01.571324 31524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-0/data/instance:
uuid: "1b7a3c3348af4646830d82af02019a66"
format_stamp: "Formatted at 2026-08-11 12:00:01 on dist-test-slave-zkpd"
server_key: "d041912f86c979ed5bdb4a6fa9e06029"
server_key_iv: "51a54d33da31c795b004b7cf6c3cfdbb"
server_key_version: "encryptionkey@0"
I20260811 12:00:01.571856 31524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-0/wal/instance:
uuid: "1b7a3c3348af4646830d82af02019a66"
format_stamp: "Formatted at 2026-08-11 12:00:01 on dist-test-slave-zkpd"
server_key: "d041912f86c979ed5bdb4a6fa9e06029"
server_key_iv: "51a54d33da31c795b004b7cf6c3cfdbb"
server_key_version: "encryptionkey@0"
I20260811 12:00:01.575778 31524 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 12:00:01.578282 31542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:00:01.579522 31524 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 12:00:01.579629 31524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-0/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-0/wal
uuid: "1b7a3c3348af4646830d82af02019a66"
format_stamp: "Formatted at 2026-08-11 12:00:01 on dist-test-slave-zkpd"
server_key: "d041912f86c979ed5bdb4a6fa9e06029"
server_key_iv: "51a54d33da31c795b004b7cf6c3cfdbb"
server_key_version: "encryptionkey@0"
I20260811 12:00:01.579721 31524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:00:01.601706 31524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:00:01.604858 31524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:00:01.605024 31524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:00:01.612317 31524 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.190:38553
I20260811 12:00:01.612627 31600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.190:38553 every 8 connection(s)
I20260811 12:00:01.613433 31524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-0/data/info.pb
I20260811 12:00:01.616418 31601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:00:01.620370 31601 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38553 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:00:01.623800 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 31524
I20260811 12:00:01.623970 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-0/wal/instance
I20260811 12:00:01.624330 6718 external_mini_cluster.cc:1428] Setting key fa6bbb05ace353c771f1604583ca4a03
Aug 11 12:00:01 dist-test-slave-zkpd krb5kdc[31511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449601, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.143.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.189@KRBTEST.COM" created.
I20260811 12:00:01.638926 31603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:01.622755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:38553 (local address 127.0.0.1:54970)
0811 12:00:01.623313 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0811 12:00:01.623356 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 12:00:01.624713 (+ 1357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:00:01.625515 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:00:01.625524 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:00:01.626110 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:00:01.626580 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:00:01.626606 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:00:01.628429 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:00:01.628432 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:00:01.629300 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:00:01.629311 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:00:01.630335 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:00:01.630338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:00:01.630440 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:01.631116 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:00:01.631135 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:00:01.632968 (+ 1833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:00:01.635302 (+ 2334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:00:01.635311 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:00:01.635321 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:00:01.635657 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:00:01.636039 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:00:01.636042 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:00:01.636043 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:00:01.636172 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:00:01.636797 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:00:01.636804 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:00:01.636996 (+ 192us) client_negotiation.cc:769] Sending connection context
0811 12:00:01.637277 (+ 281us) client_negotiation.cc:240] Negotiation successful
0811 12:00:01.637887 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":175,"thread_start_us":117,"threads_started":1}
I20260811 12:00:01.638943 31604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:01.622756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54970 (local address 127.6.143.190:38553)
0811 12:00:01.623273 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:01.623291 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:01.623480 (+ 189us) server_negotiation.cc:407] Connection header received
0811 12:00:01.625077 (+ 1597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:01.625082 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:01.625281 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:01.625438 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:00:01.626808 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:01.628277 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:01.629448 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:01.630157 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:01.630354 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:01.633135 (+ 2781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:00:01.633147 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:00:01.633163 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:00:01.633193 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:00:01.635172 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:00:01.635768 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:00:01.635773 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:00:01.635774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:00:01.635884 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:00:01.636263 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:00:01.636265 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:00:01.636266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:00:01.636661 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:00:01.636768 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 12:00:01.637374 (+ 606us) server_negotiation.cc:299] Negotiation successful
0811 12:00:01.637770 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":391,"spinlock_wait_cycles":6528,"thread_start_us":217,"threads_started":1}
I20260811 12:00:01.643005 31601 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.189" port: 39709 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:00:01.643959 31603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:01.643587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:39709 (local address 127.0.0.1:39806)
0811 12:00:01.643738 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 12:00:01.643820 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.189:39709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260811 12:00:01.644176 31543 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.143.189:39709: connect: Connection refused (error 111)
I20260811 12:00:01.645120 31603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:01.644885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:39709 (local address 127.0.0.1:39810)
0811 12:00:01.645010 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 12:00:01.645063 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.189:39709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 12:00:01.645305 31601 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.143.189:39709: Network error: Client connection negotiation failed: client connection to 127.6.143.189:39709: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.189.keytab.
Entry for principal kudu/127.6.143.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.189.keytab.
I20260811 12:00:01.655012 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.189:39709
--webserver_interface=127.6.143.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.189.keytab
--principal=kudu/127.6.143.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:41533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:38553,127.6.143.189:39709,127.6.143.188:41291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:00:01.694558 31601 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.143.189" port: 39709 } attempt: 1
I20260811 12:00:01.695536 31603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:01.695150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:39709 (local address 127.0.0.1:39814)
0811 12:00:01.695342 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 12:00:01.695427 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.189:39709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 12:00:01.696405 31603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:01.696148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:39709 (local address 127.0.0.1:39826)
0811 12:00:01.696271 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 12:00:01.696331 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.189:39709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 12:00:01.696553 31601 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.143.189:39709: Network error: Client connection negotiation failed: client connection to 127.6.143.189:39709: connect: Connection refused (error 111)
I20260811 12:00:01.764729 31601 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.143.189" port: 39709 } attempt: 2
I20260811 12:00:01.765658 31603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:01.765287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:39709 (local address 127.0.0.1:39830)
0811 12:00:01.765473 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 12:00:01.765557 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.189:39709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 12:00:01.766436 31603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:01.766178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:39709 (local address 127.0.0.1:39834)
0811 12:00:01.766308 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 12:00:01.766368 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.189:39709: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 12:00:01.766574 31601 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.143.189:39709: Network error: Client connection negotiation failed: client connection to 127.6.143.189:39709: connect: Connection refused (error 111)
W20260811 12:00:01.785385 31610 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 12:00:01.785754 31610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:00:01.785817 31610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:00:01.786005 31610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:00:01.790427 31610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:00:01.790486 31610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:00:01.790505 31610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:00:01.790549 31610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:00:01.790585 31610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:00:01.796043 31610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:41533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:38553,127.6.143.189:39709,127.6.143.188:41291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.189.keytab
--principal=kudu/127.6.143.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.189:39709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.143.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31610
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 12:00:01.797156 31610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:00:01.798208 31610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:00:01.807076 31618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:00:01.807120 31615 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:00:01.807112 31610 server_base.cc:1034] running on GCE node
W20260811 12:00:01.807399 31616 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:00:01.808097 31610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:00:01.809265 31610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:00:01.810433 31610 hybrid_clock.cc:648] HybridClock initialized: now 1786449601810395 us; error 55 us; skew 500 ppm
Aug 11 12:00:01 dist-test-slave-zkpd krb5kdc[31511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449601, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:00:01.813330 31610 init.cc:377] Logged in from keytab as kudu/127.6.143.189@KRBTEST.COM (short username kudu)
I20260811 12:00:01.814280 31610 webserver.cc:460] Webserver started at http://127.6.143.189:33949/ using document root <none> and password file <none>
I20260811 12:00:01.814827 31610 fs_manager.cc:359] Metadata directory not provided
I20260811 12:00:01.814888 31610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:00:01.815066 31610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:00:01.817178 31610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-1/data/instance:
uuid: "eb45a1d05d2143f2a89a1e3846652582"
format_stamp: "Formatted at 2026-08-11 12:00:01 on dist-test-slave-zkpd"
server_key: "84b82e6f299039af3d0bb44cd1d3cfde"
server_key_iv: "2a96aec23b1c15932a553f33fa57418d"
server_key_version: "encryptionkey@0"
I20260811 12:00:01.817704 31610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-1/wal/instance:
uuid: "eb45a1d05d2143f2a89a1e3846652582"
format_stamp: "Formatted at 2026-08-11 12:00:01 on dist-test-slave-zkpd"
server_key: "84b82e6f299039af3d0bb44cd1d3cfde"
server_key_iv: "2a96aec23b1c15932a553f33fa57418d"
server_key_version: "encryptionkey@0"
I20260811 12:00:01.821470 31610 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 12:00:01.823992 31626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:00:01.825242 31610 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 12:00:01.825388 31610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-1/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-1/wal
uuid: "eb45a1d05d2143f2a89a1e3846652582"
format_stamp: "Formatted at 2026-08-11 12:00:01 on dist-test-slave-zkpd"
server_key: "84b82e6f299039af3d0bb44cd1d3cfde"
server_key_iv: "2a96aec23b1c15932a553f33fa57418d"
server_key_version: "encryptionkey@0"
I20260811 12:00:01.825505 31610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:00:01.837361 31610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:00:01.840548 31610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:00:01.840760 31610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:00:01.847882 31610 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.189:39709
I20260811 12:00:01.848258 31682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.189:39709 every 8 connection(s)
I20260811 12:00:01.848976 31610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-1/data/info.pb
I20260811 12:00:01.852070 31683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:00:01.852747 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 31610
I20260811 12:00:01.852881 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-1/wal/instance
I20260811 12:00:01.853199 6718 external_mini_cluster.cc:1428] Setting key ae92044503ba138517219e66fbf9e5f4
I20260811 12:00:01.856276 31683 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38553 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:00:01.857671 31601 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.143.189" port: 39709 } attempt: 3
WARNING: no policy specified for kudu/127.6.143.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.143.188@KRBTEST.COM" created.
Aug 11 12:00:01 dist-test-slave-zkpd krb5kdc[31511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449601, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for kudu/127.6.143.189@KRBTEST.COM
Aug 11 12:00:01 dist-test-slave-zkpd krb5kdc[31511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449601, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.189@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 12:00:01.872629 31604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:01.858994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54972 (local address 127.6.143.190:38553)
0811 12:00:01.859131 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:01.859135 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:01.860577 (+ 1442us) server_negotiation.cc:407] Connection header received
0811 12:00:01.861029 (+ 452us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:01.861033 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:01.861076 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:01.861201 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:00:01.862463 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:01.863296 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:01.864553 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:01.864896 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:01.865010 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:01.868161 (+ 3151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:00:01.868168 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:00:01.868174 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:00:01.868211 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:00:01.870611 (+ 2400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:00:01.871123 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:00:01.871127 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:00:01.871129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:00:01.871252 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:00:01.871592 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:00:01.871595 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:00:01.871597 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:00:01.871788 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:00:01.871897 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 12:00:01.872381 (+ 484us) server_negotiation.cc:299] Negotiation successful
0811 12:00:01.872477 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 12:00:01.872629 31603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:01.858425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:39709 (local address 127.0.0.1:39848)
0811 12:00:01.858610 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 12:00:01.858628 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 12:00:01.858767 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:00:01.861220 (+ 2453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:00:01.861225 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:00:01.861458 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:00:01.861604 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:00:01.861614 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:00:01.863092 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:00:01.863097 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:00:01.863803 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:00:01.863815 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:00:01.864761 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:00:01.864765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:00:01.864865 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:01.865650 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:00:01.865687 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:00:01.867262 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:00:01.870221 (+ 2959us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:00:01.870224 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:00:01.870226 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:00:01.870699 (+ 473us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:00:01.871063 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:00:01.871070 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:00:01.871075 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:00:01.871188 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:00:01.872080 (+ 892us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:00:01.872083 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:00:01.872174 (+ 91us) client_negotiation.cc:769] Sending connection context
0811 12:00:01.872285 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 12:00:01.872433 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 12:00:01.873768 31687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:01.859155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:38553 (local address 127.0.0.1:54972)
0811 12:00:01.860393 (+ 1238us) negotiation.cc:107] Waiting for socket to connect
0811 12:00:01.860465 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 12:00:01.860805 (+ 340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:00:01.861622 (+ 817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:00:01.861626 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:00:01.862081 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:00:01.862283 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:00:01.862296 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:00:01.863494 (+ 1198us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:00:01.863498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:00:01.864400 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:00:01.864411 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:00:01.865079 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:00:01.865085 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:00:01.865199 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:01.865723 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:00:01.865749 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:00:01.867967 (+ 2218us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:00:01.870728 (+ 2761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:00:01.870737 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:00:01.870748 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:00:01.871040 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:00:01.871376 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:00:01.871379 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:00:01.871380 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:00:01.871484 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:00:01.871923 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:00:01.871932 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:00:01.872113 (+ 181us) client_negotiation.cc:769] Sending connection context
0811 12:00:01.872814 (+ 701us) client_negotiation.cc:240] Negotiation successful
0811 12:00:01.872994 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1009,"mutex_wait_us":42,"spinlock_wait_cycles":3840,"thread_start_us":182,"threads_started":1}
I20260811 12:00:01.873775 31686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:01.859154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39848 (local address 127.6.143.189:39709)
0811 12:00:01.859806 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:01.859828 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:01.859876 (+ 48us) server_negotiation.cc:407] Connection header received
0811 12:00:01.860771 (+ 895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:01.860790 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:01.861046 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:01.861238 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:00:01.861769 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:01.862969 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:01.863972 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:01.864620 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:01.864746 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:01.867434 (+ 2688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:00:01.867444 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:00:01.867469 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:00:01.867505 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:00:01.870064 (+ 2559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:00:01.870799 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:00:01.870808 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:00:01.870811 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:00:01.870932 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:00:01.871319 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:00:01.871326 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:00:01.871330 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:00:01.871947 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:00:01.872085 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 12:00:01.872427 (+ 342us) server_negotiation.cc:299] Negotiation successful
0811 12:00:01.872939 (+ 512us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":485,"thread_start_us":130,"threads_started":1}
I20260811 12:00:01.875072 31683 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.189" port: 39709 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:00:01.876310 31601 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.188" port: 41291 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:00:01.876976 31603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:01.876740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:41291 (local address 127.0.0.1:55998)
0811 12:00:01.876858 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 12:00:01.876912 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:41291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 12:00:01.877674 31603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:01.877415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:41291 (local address 127.0.0.1:56006)
0811 12:00:01.877550 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 12:00:01.877608 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:41291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
W20260811 12:00:01.877836 31601 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.143.188:41291: Network error: Client connection negotiation failed: client connection to 127.6.143.188:41291: connect: Connection refused (error 111)
Aug 11 12:00:01 dist-test-slave-zkpd krb5kdc[31511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449601, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.189@KRBTEST.COM for kudu/127.6.143.189@KRBTEST.COM
I20260811 12:00:01.884096 31687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:01.875557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:39709 (local address 127.0.0.1:39856)
0811 12:00:01.875776 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0811 12:00:01.875790 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 12:00:01.875862 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:00:01.876603 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:00:01.876609 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:00:01.876790 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:00:01.876913 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:00:01.876924 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:00:01.877830 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:00:01.877833 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:00:01.878407 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:00:01.878416 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:00:01.878946 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:00:01.878949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:00:01.879027 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:01.879509 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:00:01.879521 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:00:01.880691 (+ 1170us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:00:01.882578 (+ 1887us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:00:01.882580 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:00:01.882582 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:00:01.882788 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:00:01.883133 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:00:01.883135 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:00:01.883136 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:00:01.883246 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:00:01.883647 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:00:01.883649 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:00:01.883727 (+ 78us) client_negotiation.cc:769] Sending connection context
0811 12:00:01.883807 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 12:00:01.883984 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.143.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.188.keytab.
Entry for principal kudu/127.6.143.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.188.keytab.
I20260811 12:00:01.884277 31686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:01.875788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39856 (local address 127.6.143.189:39709)
0811 12:00:01.876304 (+ 516us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:01.876311 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:01.876328 (+ 17us) server_negotiation.cc:407] Connection header received
0811 12:00:01.876363 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:01.876365 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:01.876416 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:01.876489 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:00:01.877053 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:01.877696 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:01.878530 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:01.878806 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:01.878889 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:01.880815 (+ 1926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:00:01.880818 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:00:01.880820 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:00:01.880841 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:00:01.882478 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:00:01.882906 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:00:01.882911 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:00:01.882913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:00:01.883041 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:00:01.883331 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:00:01.883334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:00:01.883335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:00:01.883500 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:00:01.883645 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 12:00:01.883863 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 12:00:01.884071 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":51}
I20260811 12:00:01.885033 31683 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.188" port: 41291 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:00:01.885298 6718 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskEMvzen/build/debug/bin/kudu
/tmp/dist-test-taskEMvzen/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.188:41291
--webserver_interface=127.6.143.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.188.keytab
--principal=kudu/127.6.143.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.143.148:41533
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.143.190:38553,127.6.143.189:39709,127.6.143.188:41291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 12:00:01.885766 31687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:01.885502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:41291 (local address 127.0.0.1:56010)
0811 12:00:01.885649 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 12:00:01.885717 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:41291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260811 12:00:01.886224 31628 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.143.188:41291: connect: Connection refused (error 111)
I20260811 12:00:01.887220 31687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:01.886937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:41291 (local address 127.0.0.1:56016)
0811 12:00:01.887097 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 12:00:01.887150 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:41291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260811 12:00:01.887415 31683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.143.188:41291: Network error: Client connection negotiation failed: client connection to 127.6.143.188:41291: connect: Connection refused (error 111)
I20260811 12:00:01.909013 31601 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.143.188" port: 41291 } attempt: 1
I20260811 12:00:01.909919 31603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:01.909573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:41291 (local address 127.0.0.1:56032)
0811 12:00:01.909751 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 12:00:01.909827 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:41291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 12:00:01.910744 31603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:01.910497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:41291 (local address 127.0.0.1:56038)
0811 12:00:01.910622 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 12:00:01.910677 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:41291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 12:00:01.910894 31601 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.143.188:41291: Network error: Client connection negotiation failed: client connection to 127.6.143.188:41291: connect: Connection refused (error 111)
I20260811 12:00:01.936702 31683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.143.188" port: 41291 } attempt: 1
I20260811 12:00:01.937652 31687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:01.937262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:41291 (local address 127.0.0.1:56042)
0811 12:00:01.937462 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 12:00:01.937548 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:41291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 12:00:01.938449 31687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:01.938205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:41291 (local address 127.0.0.1:56054)
0811 12:00:01.938322 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 12:00:01.938375 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:41291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 12:00:01.938633 31683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.143.188:41291: Network error: Client connection negotiation failed: client connection to 127.6.143.188:41291: connect: Connection refused (error 111)
I20260811 12:00:01.986069 31601 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.143.188" port: 41291 } attempt: 2
I20260811 12:00:01.987017 31603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:01.986668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:41291 (local address 127.0.0.1:56064)
0811 12:00:01.986842 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0811 12:00:01.986920 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:41291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260811 12:00:01.987813 31603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:01.987565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:41291 (local address 127.0.0.1:56066)
0811 12:00:01.987683 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 12:00:01.987741 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:41291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 12:00:01.987991 31601 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.143.188:41291: Network error: Client connection negotiation failed: client connection to 127.6.143.188:41291: connect: Connection refused (error 111)
W20260811 12:00:02.002215 31690 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 12:00:02.002475 31690 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 12:00:02.002545 31690 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 12:00:02.002616 31690 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 12:00:02.006603 31690 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 12:00:02.006655 31690 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 12:00:02.006675 31690 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 12:00:02.006719 31690 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 12:00:02.006754 31690 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 12:00:02.006835 31683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.143.188" port: 41291 } attempt: 2
I20260811 12:00:02.007666 31687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:02.007366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:41291 (local address 127.0.0.1:56068)
0811 12:00:02.007536 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 12:00:02.007596 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:41291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 12:00:02.008492 31687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:02.008235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:41291 (local address 127.0.0.1:56072)
0811 12:00:02.008381 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 12:00:02.008435 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.143.188:41291: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260811 12:00:02.008688 31683 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.143.188:41291: Network error: Client connection negotiation failed: client connection to 127.6.143.188:41291: connect: Connection refused (error 111)
I20260811 12:00:02.012538 31690 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.143.148:41533
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.143.190:38553,127.6.143.189:39709,127.6.143.188:41291
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/krb5kdc/kudu_127.6.143.188.keytab
--principal=kudu/127.6.143.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.143.188:41291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.143.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31690
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 11:09:46 UTC on ccycloud.quasar-pllopi.root.comops.site
build id 4580136
I20260811 12:00:02.013545 31690 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 12:00:02.014575 31690 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 12:00:02.022980 31696 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:00:02.022994 31698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 12:00:02.022994 31695 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 12:00:02.023331 31690 server_base.cc:1034] running on GCE node
I20260811 12:00:02.023957 31690 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 12:00:02.025070 31690 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 12:00:02.026237 31690 hybrid_clock.cc:648] HybridClock initialized: now 1786449602026222 us; error 29 us; skew 500 ppm
Aug 11 12:00:02 dist-test-slave-zkpd krb5kdc[31511](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449602, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 12:00:02.029143 31690 init.cc:377] Logged in from keytab as kudu/127.6.143.188@KRBTEST.COM (short username kudu)
I20260811 12:00:02.030175 31690 webserver.cc:460] Webserver started at http://127.6.143.188:42483/ using document root <none> and password file <none>
I20260811 12:00:02.030759 31690 fs_manager.cc:359] Metadata directory not provided
I20260811 12:00:02.030854 31690 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 12:00:02.031093 31690 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 12:00:02.033226 31690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-2/data/instance:
uuid: "fb4a929ba74a440d8bce94237f5bf9ac"
format_stamp: "Formatted at 2026-08-11 12:00:02 on dist-test-slave-zkpd"
server_key: "13f12ea501e1258002c09d32bf6d77b7"
server_key_iv: "9568fbc160fe9d8053ea9c02df51bf8c"
server_key_version: "encryptionkey@0"
I20260811 12:00:02.033812 31690 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-2/wal/instance:
uuid: "fb4a929ba74a440d8bce94237f5bf9ac"
format_stamp: "Formatted at 2026-08-11 12:00:02 on dist-test-slave-zkpd"
server_key: "13f12ea501e1258002c09d32bf6d77b7"
server_key_iv: "9568fbc160fe9d8053ea9c02df51bf8c"
server_key_version: "encryptionkey@0"
I20260811 12:00:02.037595 31690 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 12:00:02.040033 31706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 12:00:02.041273 31690 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 12:00:02.041435 31690 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-2/data,/tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-2/wal
uuid: "fb4a929ba74a440d8bce94237f5bf9ac"
format_stamp: "Formatted at 2026-08-11 12:00:02 on dist-test-slave-zkpd"
server_key: "13f12ea501e1258002c09d32bf6d77b7"
server_key_iv: "9568fbc160fe9d8053ea9c02df51bf8c"
server_key_version: "encryptionkey@0"
I20260811 12:00:02.041549 31690 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 12:00:02.055682 31690 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 12:00:02.058651 31690 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 12:00:02.058864 31690 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 12:00:02.066385 31690 rpc_server.cc:307] RPC server started. Bound to: 127.6.143.188:41291
I20260811 12:00:02.066720 31760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.143.188:41291 every 8 connection(s)
I20260811 12:00:02.067409 31690 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-2/data/info.pb
I20260811 12:00:02.070338 31761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 12:00:02.073019 6718 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskEMvzen/build/debug/bin/kudu as pid 31690
I20260811 12:00:02.073160 6718 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskEMvzen/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786449319379472-6718-0/minicluster-data/master-2/wal/instance
I20260811 12:00:02.073460 6718 external_mini_cluster.cc:1428] Setting key 39db048f2bcb0faa28eab71895475d9d
I20260811 12:00:02.074560 31761 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38553 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:00:02 dist-test-slave-zkpd krb5kdc[31511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449602, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.188@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
Aug 11 12:00:02 dist-test-slave-zkpd krb5kdc[31511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.188@KRBTEST.COM
I20260811 12:00:02.087186 31601 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.143.188" port: 41291 } attempt: 3
I20260811 12:00:02.088096 31604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:02.076403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54982 (local address 127.6.143.190:38553)
0811 12:00:02.076553 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:02.076559 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:02.076902 (+ 343us) server_negotiation.cc:407] Connection header received
0811 12:00:02.077142 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:02.077146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:02.077202 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:02.077289 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:00:02.078177 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.078932 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.080368 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.080727 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.080839 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:02.083869 (+ 3030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:00:02.083872 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:00:02.083874 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:00:02.083935 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:00:02.085675 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:00:02.086217 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:00:02.086221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:00:02.086223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:00:02.086309 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:00:02.086744 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:00:02.086750 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:00:02.086753 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:00:02.087009 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:00:02.087131 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 12:00:02.087792 (+ 661us) server_negotiation.cc:299] Negotiation successful
0811 12:00:02.087965 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 12:00:02.089381 31765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:02.076326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.190:38553 (local address 127.0.0.1:54982)
0811 12:00:02.076762 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0811 12:00:02.076813 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 12:00:02.077006 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:00:02.077383 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:00:02.077389 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:00:02.077851 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:00:02.078035 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:00:02.078045 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.079057 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.079060 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:00:02.080253 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:00:02.080266 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.080901 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.080911 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:00:02.081069 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:02.081700 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:00:02.081726 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:00:02.083699 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:00:02.085795 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:00:02.085807 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:00:02.085817 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:00:02.086110 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:00:02.086443 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:00:02.086447 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:00:02.086452 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:00:02.086633 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:00:02.087118 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:00:02.087123 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:00:02.087445 (+ 322us) client_negotiation.cc:769] Sending connection context
0811 12:00:02.088324 (+ 879us) client_negotiation.cc:240] Negotiation successful
0811 12:00:02.088616 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"spinlock_wait_cycles":768,"thread_start_us":158,"threads_started":1}
I20260811 12:00:02.089385 31764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:02.074808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56088 (local address 127.6.143.188:41291)
0811 12:00:02.075244 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:02.075270 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:02.075306 (+ 36us) server_negotiation.cc:407] Connection header received
0811 12:00:02.076009 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:02.076033 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:02.076479 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:02.076825 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:00:02.077788 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.079157 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.080347 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.080928 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.081062 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:02.084722 (+ 3660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:00:02.084727 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:00:02.084738 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:00:02.084759 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:00:02.086279 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:00:02.086804 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:00:02.086808 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:00:02.086809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:00:02.086872 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:00:02.087211 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:00:02.087214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:00:02.087216 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:00:02.087582 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:00:02.088527 (+ 945us) server_negotiation.cc:1035] Waiting for connection context
0811 12:00:02.088634 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 12:00:02.088780 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"spinlock_wait_cycles":768,"thread_start_us":175,"threads_started":1}
I20260811 12:00:02.090938 31761 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.189" port: 39709 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 12:00:02 dist-test-slave-zkpd krb5kdc[31511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449601, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.190@KRBTEST.COM for kudu/127.6.143.188@KRBTEST.COM
Aug 11 12:00:02 dist-test-slave-zkpd krb5kdc[31511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449602, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.188@KRBTEST.COM for kudu/127.6.143.189@KRBTEST.COM
Aug 11 12:00:02 dist-test-slave-zkpd krb5kdc[31511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.190@KRBTEST.COM
I20260811 12:00:02.098374 31603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:02.087599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:41291 (local address 127.0.0.1:56090)
0811 12:00:02.087800 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 12:00:02.087814 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 12:00:02.087914 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:00:02.088426 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:00:02.088430 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:00:02.088602 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:00:02.088711 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:00:02.088718 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.089821 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.089825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:00:02.090530 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:00:02.090541 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.091135 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.091138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:00:02.091222 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:02.091740 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:00:02.091757 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:00:02.093378 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:00:02.096755 (+ 3377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:00:02.096760 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:00:02.096762 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:00:02.096974 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:00:02.097270 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:00:02.097274 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:00:02.097276 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:00:02.097373 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:00:02.097834 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:00:02.097837 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:00:02.097963 (+ 126us) client_negotiation.cc:769] Sending connection context
0811 12:00:02.098052 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 12:00:02.098186 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128}
I20260811 12:00:02.099100 31766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:02.087678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56090 (local address 127.6.143.188:41291)
0811 12:00:02.088169 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:02.088180 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:02.088195 (+ 15us) server_negotiation.cc:407] Connection header received
0811 12:00:02.088234 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:02.088238 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:02.088289 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:02.088826 (+ 537us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:00:02.088916 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.089646 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.090648 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.091004 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.091113 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:02.093503 (+ 2390us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:00:02.093508 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:00:02.093514 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:00:02.093542 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:00:02.096642 (+ 3100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:00:02.097078 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:00:02.097082 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:00:02.097084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:00:02.097173 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:00:02.097463 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:00:02.097467 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:00:02.097469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:00:02.097707 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:00:02.098680 (+ 973us) server_negotiation.cc:1035] Waiting for connection context
0811 12:00:02.098758 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 12:00:02.098875 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":98,"threads_started":1}
I20260811 12:00:02.099871 31683 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.143.188" port: 41291 } attempt: 3
I20260811 12:00:02.101838 31686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:02.091571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39870 (local address 127.6.143.189:39709)
0811 12:00:02.091708 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:02.091713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:02.091879 (+ 166us) server_negotiation.cc:407] Connection header received
0811 12:00:02.092128 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:02.092132 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:02.092186 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:02.092270 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:00:02.092899 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.093632 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.094561 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.094896 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.095023 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:02.097825 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:00:02.097829 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:00:02.097832 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:00:02.097860 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:00:02.099621 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:00:02.100453 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:00:02.100457 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:00:02.100458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:00:02.100549 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:00:02.100894 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:00:02.100898 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:00:02.100901 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:00:02.101132 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:00:02.101213 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 12:00:02.101553 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 12:00:02.101676 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 12:00:02.102542 31604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:02.092797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54986 (local address 127.6.143.190:38553)
0811 12:00:02.092897 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:02.092910 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:02.093018 (+ 108us) server_negotiation.cc:407] Connection header received
0811 12:00:02.093181 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:02.093184 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:02.093233 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:02.093314 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:00:02.094087 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.094588 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.095540 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.095797 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.095933 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:02.098767 (+ 2834us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:00:02.098771 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:00:02.098773 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:00:02.098795 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:00:02.100628 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:00:02.101108 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:00:02.101111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:00:02.101113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:00:02.101199 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:00:02.101538 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:00:02.101543 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:00:02.101546 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:00:02.101748 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:00:02.101912 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 12:00:02.102281 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 12:00:02.102394 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 12:00:02.102837 31765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:02.091453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.189:39709 (local address 127.0.0.1:39870)
0811 12:00:02.091801 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0811 12:00:02.091816 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:00:02.091978 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:00:02.092446 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:00:02.092449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:00:02.092615 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:00:02.092741 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:00:02.092748 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.093763 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.093766 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:00:02.094444 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:00:02.094454 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.095035 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.095039 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:00:02.095150 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:02.096066 (+ 916us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:00:02.096080 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:00:02.097684 (+ 1604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:00:02.100078 (+ 2394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:00:02.100081 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:00:02.100083 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:00:02.100333 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:00:02.100669 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:00:02.100672 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:00:02.100674 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:00:02.100777 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:00:02.101277 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:00:02.101282 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:00:02.101372 (+ 90us) client_negotiation.cc:769] Sending connection context
0811 12:00:02.102336 (+ 964us) client_negotiation.cc:240] Negotiation successful
0811 12:00:02.102476 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276}
I20260811 12:00:02.102872 31601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b7a3c3348af4646830d82af02019a66: Bootstrap starting.
Aug 11 12:00:02 dist-test-slave-zkpd krb5kdc[31511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449601, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.189@KRBTEST.COM for kudu/127.6.143.188@KRBTEST.COM
I20260811 12:00:02.106046 31601 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1b7a3c3348af4646830d82af02019a66: Neither blocks nor log segments found. Creating new log.
I20260811 12:00:02.107015 31601 log.cc:824] T 00000000000000000000000000000000 P 1b7a3c3348af4646830d82af02019a66: Log is configured to *not* fsync() on all Append() calls
I20260811 12:00:02.108093 31761 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.143.188" port: 41291 } has no permanent_uuid. Determining permanent_uuid...
I20260811 12:00:02.109661 31687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:02.100309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:41291 (local address 127.0.0.1:56092)
0811 12:00:02.100495 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0811 12:00:02.100510 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 12:00:02.100579 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:00:02.101108 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:00:02.101111 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:00:02.101300 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:00:02.101389 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:00:02.101396 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.102510 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.102514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:00:02.103346 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:00:02.103358 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.103971 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.103975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:00:02.104059 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:02.104539 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:00:02.104558 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:00:02.105826 (+ 1268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:00:02.107868 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:00:02.107872 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:00:02.107875 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:00:02.108179 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:00:02.108532 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:00:02.108538 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:00:02.108543 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:00:02.108642 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:00:02.109131 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:00:02.109135 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:00:02.109248 (+ 113us) client_negotiation.cc:769] Sending connection context
0811 12:00:02.109335 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 12:00:02.109473 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117}
I20260811 12:00:02.109695 31601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1b7a3c3348af4646830d82af02019a66: No bootstrap required, opened a new log
Aug 11 12:00:02 dist-test-slave-zkpd krb5kdc[31511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449601, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.143.189@KRBTEST.COM
I20260811 12:00:02.110288 31766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:02.100778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56092 (local address 127.6.143.188:41291)
0811 12:00:02.100893 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:02.100896 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:02.100908 (+ 12us) server_negotiation.cc:407] Connection header received
0811 12:00:02.100939 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:02.100944 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:02.100987 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:02.101499 (+ 512us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:00:02.101578 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.102379 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.103501 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.103812 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.103963 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:02.105976 (+ 2013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:00:02.105983 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:00:02.105986 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:00:02.106013 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:00:02.107724 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:00:02.108320 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:00:02.108323 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:00:02.108325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:00:02.108411 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:00:02.108766 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:00:02.108769 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:00:02.108771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:00:02.108996 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:00:02.109939 (+ 943us) server_negotiation.cc:1035] Waiting for connection context
0811 12:00:02.110009 (+ 70us) server_negotiation.cc:299] Negotiation successful
0811 12:00:02.110128 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 12:00:02.113579 31683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb45a1d05d2143f2a89a1e3846652582: Bootstrap starting.
Aug 11 12:00:02 dist-test-slave-zkpd krb5kdc[31511](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786449602, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.143.188@KRBTEST.COM for kudu/127.6.143.188@KRBTEST.COM
I20260811 12:00:02.114632 31601 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1b7a3c3348af4646830d82af02019a66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7a3c3348af4646830d82af02019a66" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38553 } } peers { permanent_uuid: "eb45a1d05d2143f2a89a1e3846652582" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 39709 } } peers { permanent_uuid: "fb4a929ba74a440d8bce94237f5bf9ac" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 41291 } }
I20260811 12:00:02.114888 31601 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1b7a3c3348af4646830d82af02019a66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b7a3c3348af4646830d82af02019a66, State: Initialized, Role: FOLLOWER
I20260811 12:00:02.115290 31601 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1b7a3c3348af4646830d82af02019a66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7a3c3348af4646830d82af02019a66" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38553 } } peers { permanent_uuid: "eb45a1d05d2143f2a89a1e3846652582" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 39709 } } peers { permanent_uuid: "fb4a929ba74a440d8bce94237f5bf9ac" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 41291 } }
I20260811 12:00:02.115463 31686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:02.104608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39882 (local address 127.6.143.189:39709)
0811 12:00:02.104743 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:02.104747 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:02.104759 (+ 12us) server_negotiation.cc:407] Connection header received
0811 12:00:02.104792 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:02.104795 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:02.104833 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:02.104904 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:00:02.105689 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.106382 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.107529 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.107816 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.107922 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:02.110863 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:00:02.110867 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:00:02.110870 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:00:02.110896 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:00:02.112611 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:00:02.113396 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:00:02.113415 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:00:02.113417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:00:02.113504 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:00:02.113862 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:00:02.113865 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:00:02.113867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:00:02.114081 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:00:02.114238 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 12:00:02.114624 (+ 386us) server_negotiation.cc:299] Negotiation successful
0811 12:00:02.114742 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 12:00:02.116711 31683 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb45a1d05d2143f2a89a1e3846652582: Neither blocks nor log segments found. Creating new log.
I20260811 12:00:02.116808 31601 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1b7a3c3348af4646830d82af02019a66 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:00:02.117688 31683 log.cc:824] T 00000000000000000000000000000000 P eb45a1d05d2143f2a89a1e3846652582: Log is configured to *not* fsync() on all Append() calls
I20260811 12:00:02.118811 31765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:02.108642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.143.188:41291 (local address 127.0.0.1:56094)
0811 12:00:02.109085 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0811 12:00:02.109099 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 12:00:02.109167 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 12:00:02.109461 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 12:00:02.109463 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 12:00:02.109614 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0811 12:00:02.109697 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:00:02.109707 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.111092 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.111095 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:00:02.111828 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 12:00:02.111840 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.112555 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.112558 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 12:00:02.112624 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:02.113052 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 12:00:02.113066 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 12:00:02.114489 (+ 1423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 12:00:02.117118 (+ 2629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:00:02.117121 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:00:02.117123 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:00:02.117359 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:00:02.117645 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 12:00:02.117648 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 12:00:02.117650 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 12:00:02.117747 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 12:00:02.118278 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 12:00:02.118281 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 12:00:02.118372 (+ 91us) client_negotiation.cc:769] Sending connection context
0811 12:00:02.118483 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 12:00:02.118627 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375}
I20260811 12:00:02.119500 6718 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 12:00:02.119431 31764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:02.108898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56094 (local address 127.6.143.188:41291)
0811 12:00:02.108992 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:02.108996 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:02.109238 (+ 242us) server_negotiation.cc:407] Connection header received
0811 12:00:02.109264 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:02.109267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:02.109304 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:02.109369 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:00:02.110074 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.110773 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.112014 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.112344 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.112453 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:02.114616 (+ 2163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:00:02.114620 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:00:02.114622 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:00:02.114643 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:00:02.116927 (+ 2284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:00:02.117465 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:00:02.117469 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:00:02.117472 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:00:02.117554 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:00:02.117918 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:00:02.117921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:00:02.117923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:00:02.118145 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:00:02.118243 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 12:00:02.119179 (+ 936us) server_negotiation.cc:299] Negotiation successful
0811 12:00:02.119289 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 12:00:02.120165 31683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb45a1d05d2143f2a89a1e3846652582: No bootstrap required, opened a new log
W20260811 12:00:02.123857 31781 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1b7a3c3348af4646830d82af02019a66: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:00:02.123966 31781 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:00:02.124189 31573 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "1b7a3c3348af4646830d82af02019a66"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54986
I20260811 12:00:02.124257 31768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b7a3c3348af4646830d82af02019a66 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7a3c3348af4646830d82af02019a66" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38553 } } peers { permanent_uuid: "eb45a1d05d2143f2a89a1e3846652582" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 39709 } } peers { permanent_uuid: "fb4a929ba74a440d8bce94237f5bf9ac" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 41291 } } }
I20260811 12:00:02.124399 31768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b7a3c3348af4646830d82af02019a66 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:00:02.124457 31573 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1b7a3c3348af4646830d82af02019a66 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 12:00:02.124581 31573 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1b7a3c3348af4646830d82af02019a66 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:00:02.122030 31524 master_runner.cc:186] Error getting master registration for 127.6.143.189:39709: OK, instance_id { permanent_uuid: "eb45a1d05d2143f2a89a1e3846652582" instance_seqno: 1786449601845934 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 12:00:02.125754 31524 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 12:00:02.125959 31573 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1b7a3c3348af4646830d82af02019a66 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7a3c3348af4646830d82af02019a66" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38553 } } peers { permanent_uuid: "eb45a1d05d2143f2a89a1e3846652582" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 39709 } } peers { permanent_uuid: "fb4a929ba74a440d8bce94237f5bf9ac" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 41291 } }
I20260811 12:00:02.126999 31573 leader_election.cc:288] T 00000000000000000000000000000000 P 1b7a3c3348af4646830d82af02019a66 [CANDIDATE]: Term 1 election: Requested vote from peers eb45a1d05d2143f2a89a1e3846652582 (127.6.143.189:39709), fb4a929ba74a440d8bce94237f5bf9ac (127.6.143.188:41291)
I20260811 12:00:02.129897 31761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb4a929ba74a440d8bce94237f5bf9ac: Bootstrap starting.
I20260811 12:00:02.132831 31761 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb4a929ba74a440d8bce94237f5bf9ac: Neither blocks nor log segments found. Creating new log.
I20260811 12:00:02.133759 31761 log.cc:824] T 00000000000000000000000000000000 P fb4a929ba74a440d8bce94237f5bf9ac: Log is configured to *not* fsync() on all Append() calls
I20260811 12:00:02.134577 31683 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb45a1d05d2143f2a89a1e3846652582 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7a3c3348af4646830d82af02019a66" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38553 } } peers { permanent_uuid: "eb45a1d05d2143f2a89a1e3846652582" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 39709 } } peers { permanent_uuid: "fb4a929ba74a440d8bce94237f5bf9ac" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 41291 } }
I20260811 12:00:02.134830 31683 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb45a1d05d2143f2a89a1e3846652582 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb45a1d05d2143f2a89a1e3846652582, State: Initialized, Role: FOLLOWER
I20260811 12:00:02.135181 31683 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb45a1d05d2143f2a89a1e3846652582 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7a3c3348af4646830d82af02019a66" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38553 } } peers { permanent_uuid: "eb45a1d05d2143f2a89a1e3846652582" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 39709 } } peers { permanent_uuid: "fb4a929ba74a440d8bce94237f5bf9ac" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 41291 } }
I20260811 12:00:02.136893 31683 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb45a1d05d2143f2a89a1e3846652582 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:00:02.137619 31655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1b7a3c3348af4646830d82af02019a66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "eb45a1d05d2143f2a89a1e3846652582"
I20260811 12:00:02.137872 31655 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb45a1d05d2143f2a89a1e3846652582 [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:00:02.139472 31655 raft_consensus.cc:2454] T 00000000000000000000000000000000 P eb45a1d05d2143f2a89a1e3846652582 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7a3c3348af4646830d82af02019a66 in term 1.
I20260811 12:00:02.140127 31543 leader_election.cc:302] T 00000000000000000000000000000000 P 1b7a3c3348af4646830d82af02019a66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b7a3c3348af4646830d82af02019a66, eb45a1d05d2143f2a89a1e3846652582; no voters:
I20260811 12:00:02.140473 31768 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1b7a3c3348af4646830d82af02019a66 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 12:00:02.140695 31768 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1b7a3c3348af4646830d82af02019a66 [term 1 LEADER]: Becoming Leader. State: Replica: 1b7a3c3348af4646830d82af02019a66, State: Running, Role: LEADER
I20260811 12:00:02.140839 31604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:02.129522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54998 (local address 127.6.143.190:38553)
0811 12:00:02.129646 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:02.129650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:02.130347 (+ 697us) server_negotiation.cc:407] Connection header received
0811 12:00:02.130572 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:02.130575 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:02.130626 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:02.130684 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:00:02.132213 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.133014 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.134560 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.134898 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.134982 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:02.136930 (+ 1948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:00:02.136933 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:00:02.136936 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:00:02.136962 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:00:02.138750 (+ 1788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:00:02.139281 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:00:02.139288 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:00:02.139293 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:00:02.139387 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:00:02.139754 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:00:02.139757 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:00:02.139759 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:00:02.140013 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:00:02.140118 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 12:00:02.140518 (+ 400us) server_negotiation.cc:299] Negotiation successful
0811 12:00:02.140648 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 12:00:02.141073 31768 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1b7a3c3348af4646830d82af02019a66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7a3c3348af4646830d82af02019a66" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38553 } } peers { permanent_uuid: "eb45a1d05d2143f2a89a1e3846652582" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 39709 } } peers { permanent_uuid: "fb4a929ba74a440d8bce94237f5bf9ac" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 41291 } }
I20260811 12:00:02.141177 31793 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb45a1d05d2143f2a89a1e3846652582 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7a3c3348af4646830d82af02019a66" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38553 } } peers { permanent_uuid: "eb45a1d05d2143f2a89a1e3846652582" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 39709 } } peers { permanent_uuid: "fb4a929ba74a440d8bce94237f5bf9ac" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 41291 } } }
I20260811 12:00:02.141322 31793 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb45a1d05d2143f2a89a1e3846652582 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:00:02.142904 31761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb4a929ba74a440d8bce94237f5bf9ac: No bootstrap required, opened a new log
I20260811 12:00:02.143006 31768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b7a3c3348af4646830d82af02019a66 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b7a3c3348af4646830d82af02019a66. Latest consensus state: current_term: 1 leader_uuid: "1b7a3c3348af4646830d82af02019a66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7a3c3348af4646830d82af02019a66" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38553 } } peers { permanent_uuid: "eb45a1d05d2143f2a89a1e3846652582" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 39709 } } peers { permanent_uuid: "fb4a929ba74a440d8bce94237f5bf9ac" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 41291 } } }
I20260811 12:00:02.143122 31768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b7a3c3348af4646830d82af02019a66 [sys.catalog]: This master's current role is: LEADER
I20260811 12:00:02.147307 31761 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb4a929ba74a440d8bce94237f5bf9ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7a3c3348af4646830d82af02019a66" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38553 } } peers { permanent_uuid: "eb45a1d05d2143f2a89a1e3846652582" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 39709 } } peers { permanent_uuid: "fb4a929ba74a440d8bce94237f5bf9ac" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 41291 } }
I20260811 12:00:02.147567 31761 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb4a929ba74a440d8bce94237f5bf9ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb4a929ba74a440d8bce94237f5bf9ac, State: Initialized, Role: FOLLOWER
I20260811 12:00:02.147924 31761 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb4a929ba74a440d8bce94237f5bf9ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7a3c3348af4646830d82af02019a66" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38553 } } peers { permanent_uuid: "eb45a1d05d2143f2a89a1e3846652582" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 39709 } } peers { permanent_uuid: "fb4a929ba74a440d8bce94237f5bf9ac" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 41291 } }
I20260811 12:00:02.149310 31761 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb4a929ba74a440d8bce94237f5bf9ac [sys.catalog]: configured and running, proceeding with master startup.
I20260811 12:00:02.149509 31733 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1b7a3c3348af4646830d82af02019a66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb4a929ba74a440d8bce94237f5bf9ac"
I20260811 12:00:02.149755 31733 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb4a929ba74a440d8bce94237f5bf9ac [term 0 FOLLOWER]: Advancing to term 1
I20260811 12:00:02.152241 31733 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fb4a929ba74a440d8bce94237f5bf9ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b7a3c3348af4646830d82af02019a66 in term 1.
I20260811 12:00:02.153138 31808 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 12:00:02.154347 31808 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 12:00:02.152482 31610 master_runner.cc:186] Error getting master registration for 127.6.143.188:41291: OK, instance_id { permanent_uuid: "fb4a929ba74a440d8bce94237f5bf9ac" instance_seqno: 1786449602064447 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 12:00:02.155773 31610 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 12:00:02.158042 31809 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb4a929ba74a440d8bce94237f5bf9ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7a3c3348af4646830d82af02019a66" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38553 } } peers { permanent_uuid: "eb45a1d05d2143f2a89a1e3846652582" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 39709 } } peers { permanent_uuid: "fb4a929ba74a440d8bce94237f5bf9ac" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 41291 } } }
I20260811 12:00:02.158177 31809 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb4a929ba74a440d8bce94237f5bf9ac [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:00:02.158397 31686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:02.129547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39888 (local address 127.6.143.189:39709)
0811 12:00:02.129739 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:02.129744 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:02.130768 (+ 1024us) server_negotiation.cc:407] Connection header received
0811 12:00:02.131114 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:02.131118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:02.131174 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:02.131234 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:00:02.133436 (+ 2202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.134182 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.136910 (+ 2728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.137290 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.137386 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:02.146271 (+ 8885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:00:02.146275 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:00:02.146278 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:00:02.146307 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:00:02.148098 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:00:02.156961 (+ 8863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:00:02.156967 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:00:02.156972 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:00:02.157068 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:00:02.157348 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:00:02.157351 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:00:02.157353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:00:02.157579 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:00:02.157702 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 12:00:02.158102 (+ 400us) server_negotiation.cc:299] Negotiation successful
0811 12:00:02.158238 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121}
I20260811 12:00:02.160933 31764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:02.131995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56098 (local address 127.6.143.188:41291)
0811 12:00:02.132119 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:02.132126 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:02.142056 (+ 9930us) server_negotiation.cc:407] Connection header received
0811 12:00:02.142231 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:02.142234 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:02.142281 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:02.142335 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:00:02.148354 (+ 6019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.149071 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.150191 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:02.150545 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:02.150648 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:02.154371 (+ 3723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 12:00:02.154375 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 12:00:02.154379 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 12:00:02.154409 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 12:00:02.156269 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:00:02.156902 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:00:02.156908 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:00:02.156913 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:00:02.157013 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 12:00:02.157345 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 12:00:02.157348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 12:00:02.157350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 12:00:02.157578 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 12:00:02.157696 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 12:00:02.160662 (+ 2966us) server_negotiation.cc:299] Negotiation successful
0811 12:00:02.160791 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 12:00:02.162669 31820 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fb4a929ba74a440d8bce94237f5bf9ac: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:00:02.162750 31820 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260811 12:00:02.163542 31804 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eb45a1d05d2143f2a89a1e3846652582: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 12:00:02.163618 31804 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 12:00:02.172749 31733 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fb4a929ba74a440d8bce94237f5bf9ac [term 1 FOLLOWER]: Refusing update from remote peer 1b7a3c3348af4646830d82af02019a66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:00:02.175791 31806 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1b7a3c3348af4646830d82af02019a66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb4a929ba74a440d8bce94237f5bf9ac" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 41291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:00:02.176811 31655 raft_consensus.cc:1261] T 00000000000000000000000000000000 P eb45a1d05d2143f2a89a1e3846652582 [term 1 FOLLOWER]: Refusing update from remote peer 1b7a3c3348af4646830d82af02019a66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 12:00:02.177529 31806 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1b7a3c3348af4646830d82af02019a66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb45a1d05d2143f2a89a1e3846652582" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 39709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 12:00:02.181784 31793 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb45a1d05d2143f2a89a1e3846652582 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b7a3c3348af4646830d82af02019a66. Latest consensus state: current_term: 1 leader_uuid: "1b7a3c3348af4646830d82af02019a66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7a3c3348af4646830d82af02019a66" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38553 } } peers { permanent_uuid: "eb45a1d05d2143f2a89a1e3846652582" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 39709 } } peers { permanent_uuid: "fb4a929ba74a440d8bce94237f5bf9ac" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 41291 } } }
I20260811 12:00:02.181838 31809 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb4a929ba74a440d8bce94237f5bf9ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1b7a3c3348af4646830d82af02019a66. Latest consensus state: current_term: 1 leader_uuid: "1b7a3c3348af4646830d82af02019a66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7a3c3348af4646830d82af02019a66" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38553 } } peers { permanent_uuid: "eb45a1d05d2143f2a89a1e3846652582" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 39709 } } peers { permanent_uuid: "fb4a929ba74a440d8bce94237f5bf9ac" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 41291 } } }
I20260811 12:00:02.181927 31793 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb45a1d05d2143f2a89a1e3846652582 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:00:02.181959 31809 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb4a929ba74a440d8bce94237f5bf9ac [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:00:02.185453 31806 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b7a3c3348af4646830d82af02019a66 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1b7a3c3348af4646830d82af02019a66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7a3c3348af4646830d82af02019a66" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38553 } } peers { permanent_uuid: "eb45a1d05d2143f2a89a1e3846652582" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 39709 } } peers { permanent_uuid: "fb4a929ba74a440d8bce94237f5bf9ac" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 41291 } } }
I20260811 12:00:02.185585 31806 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b7a3c3348af4646830d82af02019a66 [sys.catalog]: This master's current role is: LEADER
I20260811 12:00:02.187085 31809 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb4a929ba74a440d8bce94237f5bf9ac [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1b7a3c3348af4646830d82af02019a66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7a3c3348af4646830d82af02019a66" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38553 } } peers { permanent_uuid: "eb45a1d05d2143f2a89a1e3846652582" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 39709 } } peers { permanent_uuid: "fb4a929ba74a440d8bce94237f5bf9ac" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 41291 } } }
I20260811 12:00:02.187217 31809 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb4a929ba74a440d8bce94237f5bf9ac [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:00:02.190533 31824 mvcc.cc:205] Tried to move back new op lower bound from 7317297570487291904 to 7317297570373517312. Current Snapshot: MvccSnapshot[applied={T|T < 7317297570487291904}]
I20260811 12:00:02.191278 31768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1b7a3c3348af4646830d82af02019a66 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1b7a3c3348af4646830d82af02019a66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7a3c3348af4646830d82af02019a66" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38553 } } peers { permanent_uuid: "eb45a1d05d2143f2a89a1e3846652582" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 39709 } } peers { permanent_uuid: "fb4a929ba74a440d8bce94237f5bf9ac" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 41291 } } }
I20260811 12:00:02.191396 31768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1b7a3c3348af4646830d82af02019a66 [sys.catalog]: This master's current role is: LEADER
I20260811 12:00:02.192436 31808 catalog_manager.cc:1347] Generated new cluster ID: de1abfa086654180a15a7057731a3570
I20260811 12:00:02.192502 31808 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 12:00:02.192566 31793 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb45a1d05d2143f2a89a1e3846652582 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1b7a3c3348af4646830d82af02019a66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1b7a3c3348af4646830d82af02019a66" member_type: VOTER last_known_addr { host: "127.6.143.190" port: 38553 } } peers { permanent_uuid: "eb45a1d05d2143f2a89a1e3846652582" member_type: VOTER last_known_addr { host: "127.6.143.189" port: 39709 } } peers { permanent_uuid: "fb4a929ba74a440d8bce94237f5bf9ac" member_type: VOTER last_known_addr { host: "127.6.143.188" port: 41291 } } }
I20260811 12:00:02.192660 31793 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb45a1d05d2143f2a89a1e3846652582 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 12:00:02.197506 31826 mvcc.cc:205] Tried to move back new op lower bound from 7317297570487291904 to 7317297570373517312. Current Snapshot: MvccSnapshot[applied={T|T < 7317297570487291904}]
I20260811 12:00:02.224323 31808 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 12:00:02.225709 31808 catalog_manager.cc:1504] Loading token signing keys...
I20260811 12:00:02.245350 31808 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1b7a3c3348af4646830d82af02019a66: Generated new TSK 0
I20260811 12:00:02.246296 31808 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 12:00:02.276298 31604 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:00:02.273544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55012 (local address 127.6.143.190:38553)
0811 12:00:02.273703 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:02.273707 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:02.275076 (+ 1369us) server_negotiation.cc:407] Connection header received
0811 12:00:02.275281 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:02.275285 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:02.275340 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:02.275394 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:00:02.276136 (+ 742us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55012: BlockingRecv error: recv got EOF from 127.0.0.1:55012 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260811 12:00:02.277747 31764 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:00:02.274435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56112 (local address 127.6.143.188:41291)
0811 12:00:02.274553 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:02.274557 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:02.276833 (+ 2276us) server_negotiation.cc:407] Connection header received
0811 12:00:02.277009 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:02.277012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:02.277062 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:02.277113 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:00:02.277622 (+ 509us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56112: BlockingRecv error: recv got EOF from 127.0.0.1:56112 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 12:00:02.279706 31686 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:00:02.278241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39904 (local address 127.6.143.189:39709)
0811 12:00:02.278354 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:02.278359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:02.278808 (+ 449us) server_negotiation.cc:407] Connection header received
0811 12:00:02.278980 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:02.278983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:02.279032 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:02.279084 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:00:02.279586 (+ 502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39904: BlockingRecv error: recv got EOF from 127.0.0.1:39904 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 12:00:02.286144 31686 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:00:02.284324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39908 (local address 127.6.143.189:39709)
0811 12:00:02.284484 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:02.284489 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:02.285078 (+ 589us) server_negotiation.cc:407] Connection header received
0811 12:00:02.285338 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:02.285341 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:02.285429 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:02.285488 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:00:02.286037 (+ 549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39908: BlockingRecv error: recv got EOF from 127.0.0.1:39908 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260811 12:00:02.291481 31686 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:00:02.290021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39916 (local address 127.6.143.189:39709)
0811 12:00:02.290125 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:02.290129 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:02.290598 (+ 469us) server_negotiation.cc:407] Connection header received
0811 12:00:02.290781 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:02.290784 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:02.290835 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:02.290886 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:00:02.291381 (+ 495us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39916: BlockingRecv error: recv got EOF from 127.0.0.1:39916 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260811 12:00:02.297219 31686 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:00:02.295715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39930 (local address 127.6.143.189:39709)
0811 12:00:02.295815 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:02.295819 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:02.296314 (+ 495us) server_negotiation.cc:407] Connection header received
0811 12:00:02.296504 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:02.296507 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:02.296557 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:02.296609 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:00:02.297102 (+ 493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39930: BlockingRecv error: recv got EOF from 127.0.0.1:39930 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260811 12:00:02.305332 31686 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:00:02.303591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39938 (local address 127.6.143.189:39709)
0811 12:00:02.303749 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:02.303753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:02.304334 (+ 581us) server_negotiation.cc:407] Connection header received
0811 12:00:02.304540 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:02.304543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:02.304598 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:02.304652 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:00:02.305215 (+ 563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39938: BlockingRecv error: recv got EOF from 127.0.0.1:39938 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
W20260811 12:00:02.317867 31686 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:00:02.316043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39950 (local address 127.6.143.189:39709)
0811 12:00:02.316171 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:02.316176 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:02.316780 (+ 604us) server_negotiation.cc:407] Connection header received
0811 12:00:02.317005 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:02.317009 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:02.317066 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:02.317122 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:00:02.317748 (+ 626us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39950: BlockingRecv error: recv got EOF from 127.0.0.1:39950 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 12:00:02.338836 31686 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:00:02.336988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39962 (local address 127.6.143.189:39709)
0811 12:00:02.337122 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:02.337127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:02.337728 (+ 601us) server_negotiation.cc:407] Connection header received
0811 12:00:02.337965 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:02.337969 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:02.338029 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:02.338089 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:00:02.338710 (+ 621us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39962: BlockingRecv error: recv got EOF from 127.0.0.1:39962 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 12:00:02.375315 31686 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:00:02.373829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39978 (local address 127.6.143.189:39709)
0811 12:00:02.373961 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:02.373966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:02.374275 (+ 309us) server_negotiation.cc:407] Connection header received
0811 12:00:02.374491 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:02.374495 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:02.374559 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:02.374651 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:00:02.375182 (+ 531us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39978: BlockingRecv error: recv got EOF from 127.0.0.1:39978 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 12:00:02.442451 31686 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:00:02.441310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39988 (local address 127.6.143.189:39709)
0811 12:00:02.441519 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:02.441523 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:02.441658 (+ 135us) server_negotiation.cc:407] Connection header received
0811 12:00:02.441738 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:02.441741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:02.441796 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:02.441878 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:00:02.442330 (+ 452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39988: BlockingRecv error: recv got EOF from 127.0.0.1:39988 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260811 12:00:02.574242 31686 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:00:02.572917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39994 (local address 127.6.143.189:39709)
0811 12:00:02.573111 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:02.573116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:02.573269 (+ 153us) server_negotiation.cc:407] Connection header received
0811 12:00:02.573464 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:02.573467 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:02.573527 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:02.573621 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:00:02.574114 (+ 493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39994: BlockingRecv error: recv got EOF from 127.0.0.1:39994 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260811 12:00:02.833923 31686 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 12:00:02.832658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40008 (local address 127.6.143.189:39709)
0811 12:00:02.832847 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:02.832852 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:02.832956 (+ 104us) server_negotiation.cc:407] Connection header received
0811 12:00:02.833148 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:02.833151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:02.833208 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:02.833301 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 12:00:02.833800 (+ 499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40008: BlockingRecv error: recv got EOF from 127.0.0.1:40008 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
I20260811 12:00:03.164191 31820 catalog_manager.cc:1259] Loaded cluster ID: de1abfa086654180a15a7057731a3570
I20260811 12:00:03.164304 31820 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fb4a929ba74a440d8bce94237f5bf9ac: loading cluster ID for follower catalog manager: success
I20260811 12:00:03.164891 31804 catalog_manager.cc:1259] Loaded cluster ID: de1abfa086654180a15a7057731a3570
I20260811 12:00:03.164955 31804 catalog_manager.cc:1552] T 00000000000000000000000000000000 P eb45a1d05d2143f2a89a1e3846652582: loading cluster ID for follower catalog manager: success
I20260811 12:00:03.166085 31820 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fb4a929ba74a440d8bce94237f5bf9ac: acquiring CA information for follower catalog manager: success
I20260811 12:00:03.166522 31804 catalog_manager.cc:1574] T 00000000000000000000000000000000 P eb45a1d05d2143f2a89a1e3846652582: acquiring CA information for follower catalog manager: success
I20260811 12:00:03.167294 31820 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fb4a929ba74a440d8bce94237f5bf9ac: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 12:00:03.167604 31804 catalog_manager.cc:1602] T 00000000000000000000000000000000 P eb45a1d05d2143f2a89a1e3846652582: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 12:00:03.352681 31896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:03.348226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40020 (local address 127.6.143.189:39709)
0811 12:00:03.348498 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:03.348503 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:03.348624 (+ 121us) server_negotiation.cc:407] Connection header received
0811 12:00:03.348814 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:03.348819 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:03.348993 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:03.349121 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:00:03.349414 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:03.350035 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:03.350949 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:03.351245 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:03.351358 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:03.351706 (+ 348us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:00:03.352241 (+ 535us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:00:03.352384 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 12:00:03.352428 (+ 44us) server_negotiation.cc:299] Negotiation successful
0811 12:00:03.352505 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":73,"threads_started":1}
I20260811 12:00:03.359153 31902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:03.355750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56126 (local address 127.6.143.188:41291)
0811 12:00:03.356012 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:03.356016 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:03.356029 (+ 13us) server_negotiation.cc:407] Connection header received
0811 12:00:03.356114 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:03.356116 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:03.356240 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:03.356340 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:00:03.356558 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:03.357062 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:03.357804 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:03.358015 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:03.358091 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:03.358330 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:00:03.358730 (+ 400us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:00:03.358823 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 12:00:03.358902 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 12:00:03.358949 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":89,"threads_started":1}
I20260811 12:00:03.366433 31902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:03.362822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56142 (local address 127.6.143.188:41291)
0811 12:00:03.362942 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:03.362947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:03.362997 (+ 50us) server_negotiation.cc:407] Connection header received
0811 12:00:03.363139 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:03.363143 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:03.363200 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:03.363270 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:00:03.363499 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:03.364128 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:03.365169 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:03.365493 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:03.365610 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:03.365896 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:00:03.365962 (+ 66us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:00:03.366046 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 12:00:03.366216 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 12:00:03.366270 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 12:00:03.366920 31896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:03.362462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40024 (local address 127.6.143.189:39709)
0811 12:00:03.362574 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:03.362578 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:03.362797 (+ 219us) server_negotiation.cc:407] Connection header received
0811 12:00:03.363291 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:03.363297 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:03.363354 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:03.363426 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:00:03.363689 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:03.364488 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:03.365670 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:03.365979 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:03.366058 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:03.366412 (+ 354us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:00:03.366487 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:00:03.366555 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 12:00:03.366725 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 12:00:03.366787 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 12:00:03.367215 31910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 12:00:03.362687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55018 (local address 127.6.143.190:38553)
0811 12:00:03.362985 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 12:00:03.362989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 12:00:03.363001 (+ 12us) server_negotiation.cc:407] Connection header received
0811 12:00:03.363080 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 12:00:03.363082 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 12:00:03.363191 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 12:00:03.363345 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 12:00:03.363756 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:03.364555 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:03.365406 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 12:00:03.365692 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 12:00:03.365810 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 12:00:03.366134 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 12:00:03.366622 (+ 488us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 12:00:03.366703 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 12:00:03.366862 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 12:00:03.367051 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":70,"threads_started":1}
I20260811 12:00:03.370535 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 31524
I20260811 12:00:03.381177 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 31610
I20260811 12:00:03.389883 6718 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskEMvzen/build/debug/bin/kudu with pid 31690
2026-08-11T12:00:03Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3204 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6340 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (284005 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST